Skip to content

Commit 4012278

Browse files
simolus3simolus3
committed
Add sqlite3_test utility
1 parent 9286be8 commit 4012278

File tree

9 files changed

+399
-1
lines changed

9 files changed

+399
-1
lines changed

‎.github/workflows/main.yml‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -173,7 +173,13 @@ jobs:
173173
dart-dependencies-${{ matrix.dart }}-
174174
dart-dependencies-
175175
176-
- name: Test
176+
- name: Test sqlite3 package
177+
run: |
178+
dart pub get
179+
dart test -P ci
180+
working-directory: sqlite3/
181+
182+
- name: Test sqlite3_test package
177183
run: |
178184
dart pub get
179185
dart test -P ci

‎sqlite3_test/.gitignore‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# https://dart.dev/guides/libraries/private-files
2+
# Created by `dart pub`
3+
.dart_tool/
4+
5+
# Avoid committing pubspec.lock for library packages; see
6+
# https://dart.dev/guides/libraries/private-files#pubspeclock.
7+
pubspec.lock

‎sqlite3_test/CHANGELOG.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
## 0.1.0
2+
3+
- Initial version.

‎sqlite3_test/README.md‎

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
This package provides utilities for accessing SQLite databases in Dart tests.
2+
3+
## Features
4+
5+
Given that SQLite has no external dependencies and runs in the process of your
6+
app, it can easily be used in unit tests (avoiding the hassle of writing mocks
7+
for your database and repositories).
8+
9+
However, being a C library, SQLite is unaware of other Dart utilities typically
10+
used in tests (like a fake time with `package:clock` or a custom file system
11+
based on `package:file`).
12+
When your database queries depend on `CURRENT_TIMESTAMP`, this makes it hard
13+
to reliably test them as `clock.now()` and `CURRENT_TIMESTAMP` would report
14+
different values.
15+
16+
As a solution, this small package makes SQLite easier to integrate into your
17+
tests by providing a [VFS](https://sqlite.org/vfs.html) that will:
18+
19+
1. Make `CURRENT_TIME`, `CURRENT_DATE` and `CURRENT_TIMESTAMP` reflect the time
20+
returned by `package:clock`.
21+
2. For IO operations, allows providing a `FileSystem` from `package:file`. This
22+
includes custom implementations and the default one respecting
23+
`IOOverrides`.
24+
25+
## Usage
26+
27+
This package is intended to be used in tests, so begin by adding a dev
28+
dependency on it:
29+
30+
```
31+
$ dart pub add --dev sqlite3_test
32+
```
33+
34+
You can then use it in tests by creating an instance of `TestSqliteFileSystem`
35+
for your databases:
36+
37+
```dart
38+
import 'package:fake_async/fake_async.dart';
39+
import 'package:sqlite3/sqlite3.dart';
40+
import 'package:sqlite3_test/sqlite3_test.dart';
41+
import 'package:file/local.dart';
42+
import 'package:test/test.dart';
43+
44+
void main() {
45+
late TestSqliteFileSystem vfs;
46+
47+
setUpAll(() {
48+
vfs = TestSqliteFileSystem(fs: const LocalFileSystem());
49+
sqlite3.registerVirtualFileSystem(vfs);
50+
});
51+
tearDownAll(() => sqlite3.unregisterVirtualFileSystem(vfs));
52+
53+
test('my test depending on database time', () {
54+
final database = sqlite3.openInMemory(vfs: vfs.name);
55+
addTearDown(database.dispose);
56+
57+
// The VFS uses package:clock to get the current time, which can be
58+
// overridden for tests:
59+
final moonLanding = DateTime.utc(1969, 7, 20, 20, 18, 04);
60+
FakeAsync(initialTime: moonLanding).run((_) {
61+
final row = database.select('SELECT unixepoch(current_timestamp)').first;
62+
63+
expect(row.columnAt(0), -14182916);
64+
});
65+
});
66+
}
67+
```
68+
69+
## Limitations
70+
71+
The layer of indirection through Dart will likely make your databases slower.
72+
For this reason, this package is intended to be used in tests (as the overhead
73+
is not a problem there).
74+
75+
Also, note that `TestSqliteFileSystem` cannot be used with WAL databases as the
76+
file system does not implement memory-mapped IO.

‎sqlite3_test/analysis_options.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
include: package:lints/recommended.yaml

‎sqlite3_test/example/sqlite3_test_example.dart‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
import 'package:fake_async/fake_async.dart';
2+
import 'package:sqlite3/sqlite3.dart';
3+
import 'package:sqlite3_test/sqlite3_test.dart';
4+
import 'package:file/local.dart';
5+
import 'package:test/test.dart';
6+
7+
void main() {
8+
late TestSqliteFileSystem vfs;
9+
10+
setUpAll(() {
11+
vfs = TestSqliteFileSystem(fs: const LocalFileSystem());
12+
sqlite3.registerVirtualFileSystem(vfs);
13+
});
14+
tearDownAll(() => sqlite3.unregisterVirtualFileSystem(vfs));
15+
16+
test('my test depending on database time', () {
17+
final database = sqlite3.openInMemory(vfs: vfs.name);
18+
addTearDown(database.dispose);
19+
20+
// The VFS uses package:clock to get the current time, which can be
21+
// overridden for tests:
22+
final moonLanding = DateTime.utc(1969, 7, 20, 20, 18, 04);
23+
FakeAsync(initialTime: moonLanding).run((_) {
24+
final row = database.select('SELECT unixepoch(current_timestamp)').first;
25+
26+
expect(row.columnAt(0), -14182916);
27+
});
28+
});
29+
}

‎sqlite3_test/lib/sqlite3_test.dart‎

Lines changed: 188 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,188 @@
1+
/// Provides a virtual filesystem implementation for SQLite based on the `file`
2+
/// and `clock` packages.
3+
///
4+
/// This makes it easier to use SQLite in tests, as SQL constructs like
5+
/// `CURRENT_TIMESTAMP` will reflect the fake time of `package:clock`, allowing
6+
/// SQL logic relying on time to be tested reliably. Additionally, using
7+
/// `dart:clock` allows testing the IO behavior of SQLite databases if
8+
/// necessary.
9+
library;
10+
11+
import 'dart:typed_data';
12+
13+
import 'package:clock/clock.dart';
14+
import 'package:file/file.dart';
15+
import 'package:sqlite3/common.dart';
16+
17+
final class TestSqliteFileSystem extends BaseVirtualFileSystem {
18+
static int _counter = 0;
19+
20+
final FileSystem _fs;
21+
Directory? _createdTmp;
22+
int _tmpFileCounter = 0;
23+
24+
TestSqliteFileSystem({required FileSystem fs, String? name})
25+
: _fs = fs,
26+
super(name: name ?? 'dart-test-vfs-${_counter++}');
27+
28+
Directory get _tempDirectory {
29+
return _createdTmp ??=
30+
_fs.systemTempDirectory.createTempSync('dart-sqlite3-test');
31+
}
32+
33+
@override
34+
int xAccess(String path, int flags) {
35+
switch (flags) {
36+
case 0:
37+
// Exists
38+
return _fs.typeSync(path) == FileSystemEntityType.file ? 1 : 0;
39+
default:
40+
// Check readable and writable
41+
try {
42+
final file = _fs.file(path);
43+
file.openSync(mode: FileMode.write).closeSync();
44+
return 1;
45+
} on IOException {
46+
return 0;
47+
}
48+
}
49+
}
50+
51+
@override
52+
DateTime xCurrentTime() {
53+
return clock.now();
54+
}
55+
56+
@override
57+
void xDelete(String path, int syncDir) {
58+
_fs.file(path).deleteSync();
59+
}
60+
61+
@override
62+
String xFullPathName(String path) {
63+
return _fs.path.absolute(path);
64+
}
65+
66+
@override
67+
XOpenResult xOpen(Sqlite3Filename path, int flags) {
68+
final fsPath = path.path ??
69+
_tempDirectory.childFile((_tmpFileCounter++).toString()).absolute.path;
70+
final type = _fs.typeSync(fsPath);
71+
72+
if (type != FileSystemEntityType.notFound &&
73+
type != FileSystemEntityType.file) {
74+
throw VfsException(ErrorCodes.EINVAL);
75+
}
76+
77+
if (flags & SqlFlag.SQLITE_OPEN_EXCLUSIVE != 0 &&
78+
type != FileSystemEntityType.notFound) {
79+
throw VfsException(ErrorCodes.EEXIST);
80+
}
81+
if (flags & SqlFlag.SQLITE_OPEN_CREATE != 0 &&
82+
type == FileSystemEntityType.notFound) {
83+
_fs.file(fsPath).createSync();
84+
}
85+
86+
final deleteOnClose = flags & SqlFlag.SQLITE_OPEN_DELETEONCLOSE != 0;
87+
final readonly = flags & SqlFlag.SQLITE_OPEN_READONLY != 0;
88+
final vsFile = _fs.file(fsPath);
89+
final file =
90+
vsFile.openSync(mode: readonly ? FileMode.read : FileMode.write);
91+
92+
return (
93+
file: _TestFile(vsFile, file, deleteOnClose),
94+
outFlags: readonly ? SqlFlag.SQLITE_OPEN_READONLY : 0,
95+
);
96+
}
97+
98+
@override
99+
void xSleep(Duration duration) {}
100+
}
101+
102+
final class _TestFile implements VirtualFileSystemFile {
103+
final File _path;
104+
final RandomAccessFile _file;
105+
final bool _deleteOnClose;
106+
int _lockLevel = SqlFileLockingLevels.SQLITE_LOCK_NONE;
107+
108+
_TestFile(this._path, this._file, this._deleteOnClose);
109+
110+
@override
111+
void xClose() {
112+
_file.closeSync();
113+
if (_deleteOnClose) {
114+
_path.deleteSync();
115+
}
116+
}
117+
118+
@override
119+
int get xDeviceCharacteristics => 0;
120+
121+
@override
122+
int xFileSize() => _file.lengthSync();
123+
124+
@override
125+
void xRead(Uint8List target, int fileOffset) {
126+
_file.setPositionSync(fileOffset);
127+
final bytesRead = _file.readIntoSync(target);
128+
if (bytesRead < target.length) {
129+
target.fillRange(bytesRead, target.length, 0);
130+
throw VfsException(SqlExtendedError.SQLITE_IOERR_SHORT_READ);
131+
}
132+
}
133+
134+
@override
135+
void xSync(int flags) {
136+
_file.flushSync();
137+
}
138+
139+
@override
140+
void xTruncate(int size) {
141+
_file.truncateSync(size);
142+
}
143+
144+
@override
145+
void xWrite(Uint8List buffer, int fileOffset) {
146+
_file
147+
..setPositionSync(fileOffset)
148+
..writeFromSync(buffer);
149+
}
150+
151+
@override
152+
int xCheckReservedLock() {
153+
// RandomAccessFile doesn't appear to expose information on whether another
154+
// process is holding locks.
155+
return _lockLevel > SqlFileLockingLevels.SQLITE_LOCK_NONE ? 1 : 0;
156+
}
157+
158+
@override
159+
void xLock(int mode) {
160+
if (_lockLevel >= mode) {
161+
return;
162+
}
163+
164+
if (_lockLevel != SqlFileLockingLevels.SQLITE_LOCK_NONE) {
165+
// We want to upgrade our lock, which we do by releasing it and then
166+
// re-obtaining it.
167+
_file.unlockSync();
168+
_lockLevel = SqlFileLockingLevels.SQLITE_LOCK_NONE;
169+
}
170+
171+
final exclusive = mode > SqlFileLockingLevels.SQLITE_LOCK_SHARED;
172+
_file.lockSync(
173+
exclusive ? FileLock.blockingExclusive : FileLock.blockingShared);
174+
_lockLevel = mode;
175+
}
176+
177+
@override
178+
void xUnlock(int mode) {
179+
if (_lockLevel < mode) {
180+
return;
181+
}
182+
183+
_file.unlockSync();
184+
if (mode != SqlFileLockingLevels.SQLITE_LOCK_NONE) {
185+
return xLock(mode);
186+
}
187+
}
188+
}

‎sqlite3_test/pubspec.yaml‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
name: sqlite3_test
2+
description: Utilities for accessing sqlite3 databases in unit tests.
3+
version: 0.1.0
4+
homepage: https://github.com/simolus3/sqlite3.dart/tree/main/sqlite3_test
5+
repository: https://github.com/simolus3/sqlite3.dart
6+
7+
environment:
8+
sdk: ^3.5.4
9+
10+
dependencies:
11+
clock: ^1.1.2
12+
file: ^7.0.1
13+
sqlite3: ^2.5.0
14+
15+
dev_dependencies:
16+
fake_async: ^1.3.2
17+
lints: ^4.0.0
18+
test: ^1.24.0
19+
test_descriptor: ^2.0.1
20+
21+
dependency_overrides:
22+
sqlite3:
23+
path: ../sqlite3

0 commit comments

Comments
 (0)