Schema 4

[PaulKinlan]

Hi,

I started to get some errors with the API about 'Schema 4 not being supported'. I have my device on the beta channel so I don't know if it's that or not.

Looking at the code, I see reference to schema 3.

Anyway, I suppose this is more of a heads-up that something might be changing on the remarkable.

P

Error uploading file to Remarkable: Error: schema version 4 not supported
07-23 20:40:07
at x$.getRootHash (file:///home/runner/workspace/node_modules/rmapi-js/dist/rmapi-js.esm.min.js:16:12297)
07-23 20:40:07
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
07-23 20:40:07
at async #q (file:///home/runner/workspace/node_modules/rmapi-js/dist/rmapi-js.esm.min.js:24:1204)
07-23 20:40:07
at async Promise.all (index 4)
07-23 20:40:07
at async #J (file:///home/runner/workspace/node_modules/rmapi-js/dist/rmapi-js.esm.min.js:25:3144)
07-23 20:40:07
at async d9.putPdf (file:///home/runner/workspace/node_modules/rmapi-js/dist/rmapi-js.esm.min.js:26:304)
07-23 20:40:07
at async uploadToRemarkable (file:///home/runner/workspace/dist/index.js:186:16)
07-23 20:40:07
at async file:///home/runner/workspace/dist/index.js:334:9

[erikbrinkman]

Thanks for the report! I am aware of this, as it's been reported on my other extension even earlier, however, this is actually the appropriate place to fix it, so it's good that it's open here.

So far I haven't been able to reverse engineer the API because my account hasn't been updated, however I didn't think to enroll in beta access. I will try when I have time to reverse engineer it, and hopefully this will be fixed soon.

[PaulKinlan]

Let me know if there is anything that I can check on my end that will help you out.

[erikbrinkman]

When I have time, hopefully turning on beta updates will help. If you want to mess around in the meantime, and want to mess around. You could set up mitmproxy, and log all the output from starting up the remarkable desktop client. You could also log uploading an epub and a PDF file. If you do this, send it security as it may contain private information. If you do that, and are potentially willing to test some experimental code, that would help!

[erikbrinkman]

@PaulKinlan with some help from another developer that is on schema 4, I managed to get what I hope is a working api. The change from 3 to 4 seems pretty small, which makes me sad that it's taken this long to track down info.

If you're willing to try #29 to see if it works for you, that'd be a huge help!

Some things to note:

1. the uploadPdf and uploadEpub calls now use a different api, which means they'll work regardless, but have no options, so they just dump the file in the root of your remarkable with default options.
2. Since this could be destructive, I would be very careful with write options, so start with things like listIds and listItems.
3. If things seem to be working, and you're willing to risk destructive changes, then I would proceed carefully:
4. first try a no-op. const [roothash, gen] = await api.raw.getRootHash(); await api.raw.putRootHash(roothash, gen);. This should just increment the storage generation without doing anything, if this succeeds, then it means we can restore with no issues.
5. if that works, then back up your root hash, since you should be able to recover everything by putting the old root hash with an updated generation, e.g. [backupHash, gen, schemaVersion] = await api.raw.getRootHash(). If anything goes wrong, [, newGen] = await api.raw.getRootHash(); await api.raw.putRootHash(backupHash, newGen); should reset your remarkable to where it was before you tried anything.

If you don't have time or aren't comfortable with this, lmk, but if so, I really appreciate it!

[PaulKinlan]

Sorry for delay - I was travelling for work, really appreciate it. I'll try and give it a go this weekendm

[ryanwashburne]

@erikbrinkman any idea on when you think you'll be able to merge #29 ? using this for a project and the schema v4 drift is causing issues for some

[erikbrinkman]

@ryanwashburne, I want to merge it, but I currently don't have any guarantees for safety, and if things are wrong it could destroy someone's account. Is your api set to v4 or do you know someone who's willing to run some tests who is?

I can think of a few solutions:

1. keep waiting on @PaulKinlan (not trying to throw you under the bus, this is generally a hobby effort so there's no expectations) or someone else to try
2. I could write up an "integration test" script that would hopefully automate the instructions above. If the test worked for me, it would hopefully work for someone else, and so that'd make it easier for someone on schema v4 to test it, at which point, I'd be willing to merge
3. add some kind of safeguard that would roll out the api update, but still block v4 unless someone opted in. This would allow you to roll it out, but I'm somewhat uneasy about the risk people could be subjected tp.

Any thoughts on these or other solutions?

[ryanwashburne]

@erikbrinkman I'd be happy to test out the changes, but I believe my device is still using the v3 schema. Is there a way to force an upgrade? Or is that a gradual rollout on the reMarkable side?

Out of the options listed, 1 sounds ideal but 2 would work too. But it does sounds like we're blocked until someone gets updated to v4 schema.

[PaulKinlan]

Sorry for the delay, I got super busy in work and haven't been able to test yet :( (should have said sooner, sorry)

[erikbrinkman]

@ryanwashburne unfortunately I do not know a way to force the update. I did enroll in the bata program, but that didn't do it. AFAIK it's a gradual rollout on the rM side, but it's very gradual. The first error I saw was reported back in April.

I'm okay with 2, but it still requires someone on v4 who's willing to risk it and has some sense of what they're doing.

no worries @PaulKinlan, this is a volunteer effort, definitely no expectations!