[WIP] Permanently delete files #24

petero-dk · 2025-06-09T22:35:51Z

WARNING!!! UNTESTED

I did a little reverse engineering of the Remarkable Application, and I am pretty sure that this is the way the windows app permanently deletes files from the remarkable cloud (and from all devices)

I have a script that updates a pdf file, and the only way right now is delete create, which ends up filling the trash folder completely.

I will run a few tests, but before I wanted to do that I wanted to see if anyone had looked at this previously and came to the same solution?

erikbrinkman

thanks for contributing / working on this. I will also say if you add this, it'd be good to add a single hash purge as well just to keep the api consistent.

erikbrinkman · 2025-06-11T00:46:11Z

src/index.ts

+    const [rootHash, generation] = await this.#getRootHash(refresh);
+    // Get the raw text of the root entry
+
+    const lines = rootHash.split("\n");


the root hash should be just a string hash. Similar to other bulk actions, you should need to get the list of entries at that hash with:

const entries = await this.raw.getEntries(rootHash)

then to do a bulk purge you just need to remove the hashes from that entry set, and create a new entry with only the remaining ones, a. la. the last part of the bulk modify actions.

This way you can take advantage of the api parsing, rather than doing the raw line manipulation yourself, which is what it looks like you're doing.

This also does the v3 check (or if it doesn't, should, so some more code you can rely on general utilities.

I was not able to figure out how to get the entries list back to the root hash, is there a function that I missed? Which is why I settled for the raw manipulation, because it is just a matter of removing lines, not recalculating anything.

just look a few lines up:

const [rootEntry, uploadRoot] = await this.raw.putEntries( "root", newEntries, ); await uploadRoot; await this.#putRootHash(rootEntry.hash, generation);

the api may not be super clear, but the rootEntry has the new root hash, and uploadRoot is a promise that resolves when everything is actually uploaded. By splitting these up it allows more efficient async operations, but since you're only doing one upload, you might as well just await uploadRoot immediately.

Ahh, now I realized why I did not go this route. the put entries also uploads the docSchema file, which the delete operation did not do, I just calls the putRootHash

I have made a new small method that creates the hashstring, to skip the putEntries so it functions just like the remarkable app. I will do some testing today on it.

leaving a comment inline, after you change

src/index.ts

…sh method, added interface definitions

erikbrinkman

some more thoughts. I'm also happy to wait until you take this out of draft. I get notified, and so I have an urge to see what's up.

erikbrinkman · 2025-06-12T00:03:42Z

src/index.ts

+    const newEntries = entries.filter(
+      (entry) => !hashes.includes(entry.hash));


I would make hashes a Set so this check is O(1) not O(n)

erikbrinkman · 2025-06-12T00:04:05Z

src/index.ts

+    const [rootHash, generation] = await this.#getRootHash(refresh);
+    // Get the raw text of the root entry
+
+    const lines = rootHash.split("\n");


leaving a comment inline, after you change

erikbrinkman · 2025-06-12T00:05:42Z

src/index.ts

  }
+
+  /** permanently delete an entry */
+  async purge(hash: string, refresh: boolean = false): Promise<void> {


I would make this have a similar return type, e.g. [SimpleEntry, string] although I left a comment on the bulk purge return type as well.

The reason I did not return the simpleentry is because it no longer exists, but it would be a simple change.

yeah, lets wait until the more important details are hashed out.

erikbrinkman · 2025-06-12T00:08:21Z

src/index.ts

+  async bulkPurge(
+    hashes: readonly string[],
+    refresh: boolean = false,
+  ): Promise<[SimpleEntry[], string]> {


the similar methods return the entries that were modified. Similarly it seems to make sense to return something relating to the entries that were actually deleted. These don't tend to return the root hash.

This API has evolved over a bit of time, so I'm open to an argument of why it helps to return something different, but baring that, I lean towards a consistent interface.

I have no strong feelings on the return type. I was thinking about what could be useful data to return in this scenario and the only thing I could come up with was the root hash.

One benefit of doing something like the the other returns is that it'll tell you which were actually removed, which would indicate that maybe things were already purged.

erikbrinkman · 2025-06-12T00:11:06Z

src/index.ts

+      const hash = this.raw.makeListHash(newEntries);
+      await this.#putRootHash(hash, generation);


from this, it looks like your setting the roothash to a raw entries file, rather than the hash to a file. I've never seen a rootHash specified this way. It may work, but if writing the entries first and then writing that hash to that file also works, I strongly lean towards that option.

I also want you to be careful since this api while somewaht being non-destructive, can be destructive if you're not careful, also ideally keep backups of any documents, and a backup of the current rootHash before attempting any of this.

In short, I'm skeptical that this is the right thing to do.

I am doing this exactly like the remarkable app does, I added fiddler to my system in order to make sure I did it the same way. I can double check and possibly make a dump of the files so there is nothing I have missed.

yeah, I guess try double checking. Does the rest of the api work as intended? If so, I can't imagine that the root hash would change from a hash to a raw file.

Looking at the beginning of your function, the rootHash is a normal hash. You pass that into raw.getEntries which reads the entries list at that hash. If that step works, then presumably rootHash should contain a hash and not the full entries list. If root hash did contain an entries file, then raw.getEntries should fail, probably at validation before even making the request since I think it checks for a hash of the appropriate length.

erikbrinkman · 2025-06-12T00:12:23Z

src/raw.ts

+   * create a list hash from a set of entries
+   * @param entries - the entries to hash
+   */
+  makeListHash(entries: RawEntry[]): string;


I don't think there's much utility in exposing this, and it doesn't rely on the state of raw, so I'd move to a utility function. It could still be placed within this file, I just wouldn't put it in the RawApi.

That said, I wouldn't make any changes until there's agreement on the rootHash, as this would be moot.

Permanently delete files

3a4d0f2

petero-dk marked this pull request as draft June 9, 2025 22:36

erikbrinkman reviewed Jun 11, 2025

View reviewed changes

refactor to use list and not direct string manipulation, added makeHa…

38078e4

…sh method, added interface definitions

erikbrinkman reviewed Jun 12, 2025

View reviewed changes

		const newEntries = entries.filter(
		(entry) => !hashes.includes(entry.hash));

		const hash = this.raw.makeListHash(newEntries);
		await this.#putRootHash(hash, generation);

[WIP] Permanently delete files #24

Are you sure you want to change the base?

[WIP] Permanently delete files #24

Uh oh!

Conversation

petero-dk commented Jun 9, 2025

Uh oh!

erikbrinkman left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

petero-dk Jun 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

erikbrinkman left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

petero-dk Jun 11, 2025 •

edited

Loading