[Fix] OPFS Potential Deadlocks

[stevensJourney]

Overview

Related to:

• intermittent error occurs when using OPFSCoopSyncVFS powersync-js#664
• OPFSCoopSyncVFS causes multi-tab deadlocks in Chrome and Safari powersync-js#785

The following issue is quite difficult to reproduce. I cannot reproduce it in this WA-SQLite demo. It can be reproduced in our PowerSync JS SDK.

When opening multiple tabs (approximately 5 or more) and rapidly cycling through the tabs and refreshing. Two possible issues can occur.

A stuck navigator lock

When the SQLite database is opened, the #requestAccessHandle is called. This method requests a navigator lock which resolves to a releaser function. It then attempts to asynchronously create a synchronous access handle for the 3 sqlite persistent files. If, for any reason, the creation of the synchronous access handles failed, the lock does not seem to be released. The open call is retried, but is stuck on the request to obtain the lock (possibly since it's already obtained).

An example of when the open is stuck waiting for a lock

I'm not exactly sure what could cause these calls to fail. Perhaps there is some browser behaviour which could cause these calls to fail.

If we try...catch this process and release the lock on error, then we no longer experience these deadlocks in our testing.

NoModificationAllowed Errors

In the above refreshing test, we often also see the follow error (which is easier to reproduce in Firefox)

Failed to execute 'createSyncAccessHandle' on 'FileSystemFileHandle': Access Handles cannot be created if there is another open Access Handle or Writable stream associated with the same file.

Again, not entirely sure why this is happening. The main theory at this point is that, for some reason, creating 1 of the 3 persistent access handles fail - but 1 or more of them might have succeeded. One the next retry we might be attempting to open an access handle which is already open.

[simolus3]

I see how this is an improvement if we're able to lock some but not all of the files.

A general question: Are we supposed to retry on failures since we're in a retryOps block? It seems to me like failing to acquire the access handles indicates a locked state that can be retried, or do we explicitly want to fail queries in this case?

[stevensJourney]

A general question: Are we supposed to retry on failures since we're in a retryOps block? It seems to me like failing to acquire the access handles indicates a locked state that can be retried, or do we explicitly want to fail queries in this case?

@simolus3 It seems like the retryOps block currently retries operations while the response code is non-zero. I can see how it might make sense to also retry if there was an error thrown in one of the async operations being awaited. I'm not sure if this is the intention of the retry logic though. Perhaps errors like this should be surfaced on this level and retried on a higher level - this is what I've added to our JS SDK in powersync-ja/powersync-js#786