-
Notifications
You must be signed in to change notification settings - Fork 6
[WIP] GRDB Support #78
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
stevensJourney
wants to merge
17
commits into
main
Choose a base branch
from
grdb
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from 4 commits
Commits
Show all changes
17 commits
Select commit
Hold shift + click to select a range
245e3d2
wip: grdb connection pool
stevensJourney 8428135
wip: grdb
stevensJourney dbd9c09
Use latest GRDB package. Update tests and queries.
stevensJourney 6f32934
wip: table update hooks
stevensJourney e2681f6
Add test for GRDB updates triggered by GRDB
stevensJourney a79ec5c
add join test
stevensJourney 8a278a1
WIP: Add GRDB demo app
stevensJourney 4002d85
demo improvements
stevensJourney 6f0e630
Table updates from PowerSync side
stevensJourney 41174b1
Use SQLite Session API for PowerSync updates.
stevensJourney 7aae6cf
Update GRDB dependency
stevensJourney e970fd4
Merge remote-tracking branch 'origin/main' into grdb
stevensJourney 76aeb1c
demo update
stevensJourney 281558a
Update README. Cleanup public APIs. WIP WatchOS.
stevensJourney 7beff16
Merge remote-tracking branch 'origin/main' into grdb
stevensJourney a81986e
Update READMEs
stevensJourney 1c1f2bb
Register extension on WatchOS
stevensJourney File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,82 @@ | ||
import PowerSyncKotlin | ||
|
||
final class SwiftSQLiteConnectionPoolAdapter: PowerSyncKotlin.SwiftPoolAdapter { | ||
let pool: SQLiteConnectionPoolProtocol | ||
|
||
init( | ||
pool: SQLiteConnectionPoolProtocol | ||
) { | ||
self.pool = pool | ||
} | ||
|
||
func getPendingUpdates() -> Set<String> { | ||
return pool.getPendingUpdates() | ||
} | ||
|
||
func __closePool() async throws { | ||
do { | ||
try pool.close() | ||
} catch { | ||
try? PowerSyncKotlin.throwPowerSyncException( | ||
exception: PowerSyncException( | ||
message: error.localizedDescription, | ||
cause: nil | ||
) | ||
) | ||
} | ||
} | ||
|
||
func __leaseRead(callback: @escaping (Any) -> Void) async throws { | ||
do { | ||
try await pool.read { pointer in | ||
callback(UInt(bitPattern: pointer)) | ||
} | ||
} catch { | ||
try? PowerSyncKotlin.throwPowerSyncException( | ||
exception: PowerSyncException( | ||
message: error.localizedDescription, | ||
cause: nil | ||
) | ||
) | ||
} | ||
} | ||
|
||
func __leaseWrite(callback: @escaping (Any) -> Void) async throws { | ||
do { | ||
try await pool.write { pointer in | ||
callback(UInt(bitPattern: pointer)) | ||
} | ||
} catch { | ||
try? PowerSyncKotlin.throwPowerSyncException( | ||
exception: PowerSyncException( | ||
message: error.localizedDescription, | ||
cause: nil | ||
) | ||
) | ||
} | ||
} | ||
|
||
func __leaseAll(callback: @escaping (Any, [Any]) -> Void) async throws { | ||
// TODO, actually use all connections | ||
do { | ||
try await pool.write { pointer in | ||
callback(UInt(bitPattern: pointer), []) | ||
} | ||
} catch { | ||
try? PowerSyncKotlin.throwPowerSyncException( | ||
exception: PowerSyncException( | ||
message: error.localizedDescription, | ||
cause: nil | ||
) | ||
) | ||
} | ||
} | ||
} | ||
|
||
extension SQLiteConnectionPoolProtocol { | ||
func toKotlin() -> PowerSyncKotlin.SwiftSQLiteConnectionPool { | ||
return PowerSyncKotlin.SwiftSQLiteConnectionPool( | ||
adapter: SwiftSQLiteConnectionPoolAdapter(pool: self) | ||
) | ||
} | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,28 @@ | ||
import Foundation | ||
|
||
/// An implementation of a connection pool providing asynchronous access to a single writer and multiple readers. | ||
/// This is the underlying pool implementation on which the higher-level PowerSync Swift SDK is built on. | ||
public protocol SQLiteConnectionPoolProtocol { | ||
func getPendingUpdates() -> Set<String> | ||
|
||
/// Calls the callback with a read-only connection temporarily leased from the pool. | ||
func read( | ||
onConnection: @Sendable @escaping (OpaquePointer) -> Void, | ||
) async throws | ||
|
||
/// Calls the callback with a read-write connection temporarily leased from the pool. | ||
func write( | ||
onConnection: @Sendable @escaping (OpaquePointer) -> Void, | ||
) async throws | ||
|
||
/// Invokes the callback with all connections leased from the pool. | ||
func withAllConnections( | ||
onConnection: @Sendable @escaping ( | ||
_ writer: OpaquePointer, | ||
_ readers: [OpaquePointer] | ||
) -> Void, | ||
) async throws | ||
|
||
/// Closes the connection pool and associated resources. | ||
func close() throws | ||
} |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @stevensJourney - it looks like you're about to need a way to iterate all available connections.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi!, yes, in order to completely satisfy our internal driver requirements, we would require this.
I haven't full scanned through the GRDB docs yet, but I haven't seen a way yet to achieve this. Is this currently possible with GRDB?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, not yet, that would be a new feature. We'd have to clarify the exact required semantics.
In particular, I wonder if the abstract pool you have to conform to is assumed to run a fixed set of available connections, or if it is allowed to close existing connections and open new ones.
leaseAll
makes sure that all future database access are impacted by the effects of the callback.To lift the ambiguity, the semantics of
leaseAll
need to be clarified.There is another related clarification that is needed: can concurrent database accesses run during the execution of
leaseAll
, or not?Maybe there are other constraints that I'm not aware of. In all cases, make sure you make the semantics crystal clear.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, GRDB has a
prepareDatabase
callback that executes code in any connection that opens, before it is made available to the rest of the application. This is where GRDB users register functions, collations, and make general connection setup:If the only goal of
leaseAll
is to execute database code early, then I would suggest replacing it with a similar pattern in the PowerSync pool protocol. This should be possible in all target languages that have a notion of closure that can be executed later. And this would void all the complex semantics questions I have asked above.Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Finally, if the goal of
leaseAll
is to callsqlite3_db_release_memory()
, thenDatabasePool
has a ready-madereleaseMemory()
method.In summary, I strongly suggest clarifying the intent behind
leaseAll
, so that we avoid the XY problem (andleaseAll
has a big XY smell).There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the detailed responses and queries.
The current implementation of
leaseAll
in this SDK assumes a lock is taken on all the current SQLite connections. The pool can be fixed or dynamically sized - the only requirement is that the lock be taken at the time of and during the request.prepareDatabase
is a good solution for executing code when connections are opened, but this does not align with the current requirements ofleaseAll
.You are very correct about the XY Problem scenario described. For context, we allow users to update the PowerSync schema after the client has been initialised. E.g. This is triggered here, We currently apply the change using a write connection and thereafter refresh the schema on the read connections - preventing any reads during this operation: which might be invalid.
If are any alternatives for ensuring the Schema is refreshed on read connections, I think we could avoid requiring this functionality for now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, thanks for the clarification 👍 When you're on the leaseAll implementation, please open a new discussion in the GRDB repo. All the building blocks are there, we just need to design a public API.