|
1 | 1 | Sharing a Database |
2 | 2 | ================== |
3 | 3 |
|
4 | | -**This guide describes a recommended setup that applies as soon as several processes want to access the same SQLite database.** It complements the [Concurrency](Concurrency.md) guide, that you should read first. |
5 | | - |
6 | | -On iOS for example, you can share database files between multiple processes by storing them in an [App Group Container](https://developer.apple.com/documentation/foundation/nsfilemanager/1412643-containerurlforsecurityapplicati). On macOS as well, several processes may want to open the same database, according to their particular sandboxing contexts. |
7 | | - |
8 | | -Accessing a shared database from several SQLite connections, from several processes, creates challenges at various levels: |
9 | | - |
10 | | -1. **Database setup** may be attempted by multiple processes, concurrently. |
11 | | -2. **SQLite** may throw `SQLITE_BUSY` errors, code 5, "database is locked". |
12 | | -3. **iOS** may kill your application with a `0xDEAD10CC` exception. |
13 | | -4. **GRDB** database observation misses changes performed by external processes. |
14 | | - |
15 | | -We'll address all of those challenges below. |
16 | | - |
17 | | -- [Use a Database Pool] |
18 | | -- [How to limit the `SQLITE_BUSY` error] |
19 | | -- [How to limit the `0xDEAD10CC` exception] |
20 | | -- [How to perform cross-process database observation] |
21 | | - |
22 | | - |
23 | | -## Use a Database Pool |
24 | | - |
25 | | -In order to access a shared database, use a [Database Pool]. It opens the database in the [WAL mode](https://www.sqlite.org/wal.html), which helps sharing a database. |
26 | | - |
27 | | -Since several processes may open the database at the same time, protect the creation of the database pool with an [NSFileCoordinator]. |
28 | | - |
29 | | -- In a process that can create and write in the database, use this sample code: |
30 | | - |
31 | | - ```swift |
32 | | - /// Returns an initialized database pool at the shared location databaseURL |
33 | | - func openSharedDatabase(at databaseURL: URL) throws -> DatabasePool { |
34 | | - let coordinator = NSFileCoordinator(filePresenter: nil) |
35 | | - var coordinatorError: NSError? |
36 | | - var dbPool: DatabasePool? |
37 | | - var dbError: Error? |
38 | | - coordinator.coordinate(writingItemAt: databaseURL, options: .forMerging, error: &coordinatorError, byAccessor: { url in |
39 | | - do { |
40 | | - dbPool = try openDatabase(at: url) |
41 | | - } catch { |
42 | | - dbError = error |
43 | | - } |
44 | | - }) |
45 | | - if let error = dbError ?? coordinatorError { |
46 | | - throw error |
47 | | - } |
48 | | - return dbPool! |
49 | | - } |
50 | | - |
51 | | - private func openDatabase(at databaseURL: URL) throws -> DatabasePool { |
52 | | - var configuration = Configuration() |
53 | | - configuration.prepareDatabase { db in |
54 | | - // Activate the persistent WAL mode so that |
55 | | - // readonly processes can access the database. |
56 | | - // |
57 | | - // See https://www.sqlite.org/walformat.html#operations_that_require_locks_and_which_locks_those_operations_use |
58 | | - // and https://www.sqlite.org/c3ref/c_fcntl_begin_atomic_write.html#sqlitefcntlpersistwal |
59 | | - if db.configuration.readonly == false { |
60 | | - var flag: CInt = 1 |
61 | | - let code = withUnsafeMutablePointer(to: &flag) { flagP in |
62 | | - sqlite3_file_control(db.sqliteConnection, nil, SQLITE_FCNTL_PERSIST_WAL, flagP) |
63 | | - } |
64 | | - guard code == SQLITE_OK else { |
65 | | - throw DatabaseError(resultCode: ResultCode(rawValue: code)) |
66 | | - } |
67 | | - } |
68 | | - } |
69 | | - let dbPool = try DatabasePool(path: databaseURL.path, configuration: configuration) |
70 | | - |
71 | | - // Perform here other database setups, such as defining |
72 | | - // the database schema with a DatabaseMigrator, and |
73 | | - // checking if the application can open the file: |
74 | | - try migrator.migrate(dbPool) |
75 | | - if try dbPool.read(migrator.hasBeenSuperseded) { |
76 | | - // Database is too recent |
77 | | - throw /* some error */ |
78 | | - } |
79 | | - |
80 | | - return dbPool |
81 | | - } |
82 | | - ``` |
83 | | - |
84 | | -- In a process that only reads in the database, use this sample code: |
85 | | - |
86 | | - ```swift |
87 | | - /// Returns an initialized database pool at the shared location databaseURL, |
88 | | - /// or nil if the database is not created yet, or does not have the required |
89 | | - /// schema version. |
90 | | - func openSharedReadOnlyDatabase(at databaseURL: URL) throws -> DatabasePool? { |
91 | | - let coordinator = NSFileCoordinator(filePresenter: nil) |
92 | | - var coordinatorError: NSError? |
93 | | - var dbPool: DatabasePool? |
94 | | - var dbError: Error? |
95 | | - coordinator.coordinate(readingItemAt: databaseURL, options: .withoutChanges, error: &coordinatorError, byAccessor: { url in |
96 | | - do { |
97 | | - dbPool = try openReadOnlyDatabase(at: url) |
98 | | - } catch { |
99 | | - dbError = error |
100 | | - } |
101 | | - }) |
102 | | - if let error = dbError ?? coordinatorError { |
103 | | - throw error |
104 | | - } |
105 | | - return dbPool |
106 | | - } |
107 | | - |
108 | | - private func openReadOnlyDatabase(at databaseURL: URL) throws -> DatabasePool? { |
109 | | - do { |
110 | | - var configuration = Configuration() |
111 | | - configuration.readonly = true |
112 | | - let dbPool = try DatabasePool(path: databaseURL.path, configuration: configuration) |
113 | | - |
114 | | - // Check here if the database schema is the expected one, |
115 | | - // for example with a DatabaseMigrator: |
116 | | - return try dbPool.read { db in |
117 | | - if try migrator.hasBeenSuperseded(db) { |
118 | | - // Database is too recent |
119 | | - return nil |
120 | | - } else if try migrator.hasCompletedMigrations(db) == false { |
121 | | - // Database is too old |
122 | | - return nil |
123 | | - } |
124 | | - return dbPool |
125 | | - } |
126 | | - } catch { |
127 | | - if FileManager.default.fileExists(atPath: databaseURL.path) { |
128 | | - throw error |
129 | | - } else { |
130 | | - return nil |
131 | | - } |
132 | | - } |
133 | | - } |
134 | | - ``` |
135 | | - |
136 | | - |
137 | | -#### The specific case of readonly connections |
138 | | - |
139 | | -Readonly connections will fail unless two extra files ending in `-shm` and `-wal` are present next to the database file ([source](https://www.sqlite.org/walformat.html#operations_that_require_locks_and_which_locks_those_operations_use)). Those files are regular companions of databases in the [WAL mode]. But they are deleted, under regular operations, when database connections are closed. Precisely speaking, they *may* be deleted: it depends on the SQLite and the operating system versions ([source](https://github.com/groue/GRDB.swift/issues/739#issuecomment-604363998)). And when they are deleted, readonly connections fail. |
140 | | - |
141 | | -The solution is to enable the "persistent WAL mode", as shown in the sample code above, by setting the [SQLITE_FCNTL_PERSIST_WAL](https://www.sqlite.org/c3ref/c_fcntl_begin_atomic_write.html#sqlitefcntlpersistwal) flag. This mode makes sure the `-shm` and `-wal` files are never deleted, and guarantees a database access to readonly connections. |
142 | | - |
143 | | - |
144 | | -## How to limit the `SQLITE_BUSY` error |
145 | | - |
146 | | -> The SQLITE_BUSY result code indicates that the database file could not be written (or in some cases read) because of concurrent activity by some other database connection, usually a database connection in a separate process. |
147 | | - |
148 | | -See https://www.sqlite.org/rescode.html#busy for more information about this error. |
149 | | - |
150 | | -If several processes want to write in the database, configure the database pool of each process that wants to write: |
151 | | - |
152 | | -```swift |
153 | | -var configuration = Configuration() |
154 | | -configuration.busyMode = .timeout(/* a TimeInterval */) |
155 | | -let dbPool = try DatabasePool(path: ..., configuration: configuration) |
156 | | -``` |
157 | | - |
158 | | -With such a setup, you may still get `SQLITE_BUSY` (5, "database is locked") errors from all write operations. They will occur if the database remains locked by another process for longer than the specified timeout. |
159 | | - |
160 | | -```swift |
161 | | -do { |
162 | | - try dbPool.write { db in ... } |
163 | | -} catch DatabaseError.SQLITE_BUSY { |
164 | | - // Another process won't let you write. Deal with it. |
165 | | -} |
166 | | -``` |
167 | | - |
168 | | -> :bulb: **Tip**: In order to be nice to other processes, measure the duration of your longest writes, and attempt at optimizing the ones that last for too long. See the [How do I monitor the duration of database statements execution?] FAQ. |
169 | | - |
170 | | - |
171 | | -## How to limit the `0xDEAD10CC` exception |
172 | | - |
173 | | -> The exception code 0xDEAD10CC indicates that an application has been terminated by the OS because it held on to a file lock or sqlite database lock during suspension. |
174 | | - |
175 | | -See https://developer.apple.com/documentation/xcode/understanding-the-exception-types-in-a-crash-report for more information about this exception. |
176 | | - |
177 | | -1. If you use SQLCipher, use SQLCipher 4+, and call the [`cipher_plaintext_header_size` pragma](https://www.zetetic.net/sqlcipher/sqlcipher-api/#cipher_plaintext_header_size) from your database preparation function: |
178 | | - |
179 | | - ```swift |
180 | | - var configuration = Configuration() |
181 | | - configuration.prepareDatabase { (db: Database) in |
182 | | - try db.usePassphrase("secret") |
183 | | - try db.execute(sql: "PRAGMA cipher_plaintext_header_size = 32") |
184 | | - } |
185 | | - let dbPool = try DatabasePool(path: ..., configuration: configuration) |
186 | | - ``` |
187 | | - |
188 | | - This will avoid this issue: https://github.com/sqlcipher/sqlcipher/issues/255. |
189 | | - |
190 | | - This will also disable a SQLCipher security feature: the salt. As described by https://www.zetetic.net/sqlcipher/design/: |
191 | | - |
192 | | - > The salt is used for key derivation and it ensures that even if two databases are created using the same password, they will not have the same encryption key. |
193 | | - |
194 | | - Applications are responsible for managing the salt themselves and providing it to SQLCipher. See https://www.zetetic.net/sqlcipher/sqlcipher-api/#cipher_plaintext_header_size for instructions. |
195 | | - |
196 | | -2. [**:fire: EXPERIMENTAL**](../README.md#what-are-experimental-features) In each process that wants to write in the database: |
197 | | - |
198 | | - Set the `observesSuspensionNotifications` configuration flag: |
199 | | - |
200 | | - ```swift |
201 | | - var configuration = Configuration() |
202 | | - configuration.observesSuspensionNotifications = true |
203 | | - let dbPool = try DatabasePool(path: ..., configuration: configuration) |
204 | | - ``` |
205 | | - |
206 | | - Post `Database.suspendNotification` when the application is about to be [suspended](https://developer.apple.com/documentation/uikit/app_and_environment/managing_your_app_s_life_cycle). You can for example post this notification from `UIApplicationDelegate.applicationDidEnterBackground(_:)`, or in the expiration handler of a [background task](https://forums.developer.apple.com/thread/85066). |
207 | | - |
208 | | - ```swift |
209 | | - @UIApplicationMain |
210 | | - class AppDelegate: UIResponder, UIApplicationDelegate { |
211 | | - func applicationDidEnterBackground(_ application: UIApplication) { |
212 | | - // Suspend databases |
213 | | - NotificationCenter.default.post(name: Database.suspendNotification, object: self) |
214 | | - } |
215 | | - } |
216 | | - ``` |
217 | | - |
218 | | - Post `Database.resumeNotification` from `UIApplicationDelegate.applicationWillEnterForeground(_:)` (or `SceneDelegate.sceneWillEnterForeground(_:)` for scene-based applications): |
219 | | - |
220 | | - ```swift |
221 | | - @UIApplicationMain |
222 | | - class AppDelegate: UIResponder, UIApplicationDelegate { |
223 | | - func applicationWillEnterForeground(_ application: UIApplication) { |
224 | | - // Resume databases |
225 | | - NotificationCenter.default.post(name: Database.resumeNotification, object: self) |
226 | | - } |
227 | | - } |
228 | | - ``` |
229 | | - |
230 | | - If the application uses the background modes supported by iOS, post `Database.resumeNotification` method from each and every background mode callback that may use the database. For example, if your application supports background fetches: |
231 | | - |
232 | | - ```swift |
233 | | - @UIApplicationMain |
234 | | - class AppDelegate: UIResponder, UIApplicationDelegate { |
235 | | - func application(_ application: UIApplication, performFetchWithCompletionHandler completionHandler: @escaping (UIBackgroundFetchResult) -> Void) { |
236 | | - // Resume databases |
237 | | - NotificationCenter.default.post(name: Database.resumeNotification, object: self) |
238 | | - // Proceed with background fetch |
239 | | - ... |
240 | | - } |
241 | | - } |
242 | | - ``` |
243 | | - |
244 | | - Suspended databases greatly reduce the odds of `0xDEAD10CC` exception occurring. If you see one in your crash logs, please open an issue! |
245 | | - |
246 | | - In exchange, you will get `SQLITE_INTERRUPT` (9) or `SQLITE_ABORT` (4) errors, with messages "Database is suspended", "Transaction was aborted", or "interrupted", for any attempt at writing in the database when it is suspended. |
247 | | - |
248 | | - You can catch those errors: |
249 | | - |
250 | | - ```swift |
251 | | - do { |
252 | | - try dbPool.write { db in ... } |
253 | | - } catch DatabaseError.SQLITE_INTERRUPT, DatabaseError.SQLITE_ABORT { |
254 | | - // Oops, the database is suspended. |
255 | | - // Maybe try again after database is resumed? |
256 | | - } |
257 | | - ``` |
258 | | - |
259 | | - |
260 | | -## How to perform cross-process database observation |
261 | | - |
262 | | -GRDB [Database Observation] features, as well as its [Combine publishers] and [RxGRDB], are not able to notify database changes performed by other processes. |
263 | | - |
264 | | -Whenever you need to notify other processes that the database has been changed, you will have to use a cross-process notification mechanism such as [NSFileCoordinator] or [CFNotificationCenterGetDarwinNotifyCenter]. |
265 | | - |
266 | | -You can trigger those notifications automatically with [DatabaseRegionObservation]: |
267 | | - |
268 | | -```swift |
269 | | -// Notify all changes made to the "player" and "team" database tables |
270 | | -let observation = DatabaseRegionObservation(tracking: Player.all(), Team.all()) |
271 | | -let observer = try observation.start(in: dbPool) { (db: Database) in |
272 | | - // Notify other processes |
273 | | -} |
274 | | - |
275 | | -// Notify all changes made to the database |
276 | | -let observation = DatabaseRegionObservation(tracking: DatabaseRegion.fullDatabase) |
277 | | -let observer = try observation.start(in: dbPool) { (db: Database) in |
278 | | - // Notify other processes |
279 | | -} |
280 | | -``` |
281 | | - |
282 | | -[Use a Database Pool]: #use-a-database-pool |
283 | | -[How to limit the `SQLITE_BUSY` error]: #how-to-limit-the-sqlite_busy-error |
284 | | -[How to limit the `0xDEAD10CC` exception]: #how-to-limit-the-0xdead10cc-exception |
285 | | -[How to perform cross-process database observation]: #how-to-perform-cross-process-database-observation |
286 | | -[Database Pool]: ../README.md#database-pools |
287 | | -[Database Observation]: ../README.md#database-changes-observation |
288 | | -[RxGRDB]: https://github.com/RxSwiftCommunity/RxGRDB |
289 | | -[NSFileCoordinator]: https://developer.apple.com/documentation/foundation/nsfilecoordinator |
290 | | -[CFNotificationCenterGetDarwinNotifyCenter]: https://developer.apple.com/documentation/corefoundation/1542572-cfnotificationcentergetdarwinnot |
291 | | -[DatabaseRegionObservation]: ../README.md#databaseregionobservation |
292 | | -[WAL mode]: https://www.sqlite.org/wal.html |
293 | | -[How do I monitor the duration of database statements execution?]: ../README.md#how-do-i-monitor-the-duration-of-database-statements-execution |
294 | | -[Combine publishers]: Combine.md |
| 4 | +This guide [has moved](https://groue.github.io/GRDB.swift/docs/6.3/documentation/grdb/databasesharing). |
0 commit comments