Skip to content

Commit 31b9cce

Browse files
mbrandonwstephencelis
mbrandonw
and
stephencelis
authored
Pre-load associations via JSON (#24)
* wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * wip * Update QueryCookbook.md * remove jsonObject * wip * json encoder * revert name-to-title rename * update snaps * Update Sources/StructuredQueriesCore/Documentation.docc/Articles/QueryCookbook.md * rename * fix docs * wip * wip * cleanup * cleanup * wip * wip * Update QueryCookbook.md --------- Co-authored-by: Stephen Celis <[email protected]>
1 parent f360aea commit 31b9cce

File tree

8 files changed

+761
-10
lines changed

8 files changed

+761
-10
lines changed

‎Sources/StructuredQueriesCore/Documentation.docc/Articles/QueryCookbook.md‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,12 @@ scopes, and decoding into custom data types.
88
The library comes with a variety of tools that allow you to define helpers for composing together
99
large and complex queries.
1010

11+
* [Reusable table queries](#Reusable-table-queries)
12+
* [Reusable column queries](#Reusable-column-queries)
13+
* [Default scopes](#Default-scopes)
14+
* [Custom selections](#Custom-selections)
15+
* [Pre-loading associations with JSON](#Pre-loading-associations-with-json)
16+
1117
### Reusable table queries
1218

1319
One can define query helpers as statics on their tables in order to facilitate using those
@@ -339,3 +345,89 @@ And a query that selects into this type can be defined like so:
339345
```
340346
}
341347
}
348+
349+
### Pre-loading associations with JSON
350+
351+
There are times you may want to load rows from a table along with the data from some associated
352+
table. For example, querying for all reminders lists along with an array of the reminders in each
353+
list. We'd like to be able to query for this data and decode it into a collection of values
354+
from the following data type:
355+
356+
```struct
357+
struct Row {
358+
let remindersList: RemindersList
359+
let reminders: [Reminder]
360+
}
361+
```
362+
363+
However, typically this requires one to make multiple SQL queries. First a query to selects all
364+
of the reminders lists:
365+
366+
```swift
367+
let remindersLists = try RemindersLists.all.execute(db)
368+
```
369+
370+
Then you execute another query to fetch all of the reminders associated with the lists just
371+
fetched:
372+
373+
```swift
374+
let reminders = try Reminder
375+
.where { $0.id.in(remindersLists.map(\.id)) }
376+
.execute(db))
377+
```
378+
379+
And then finally you need to transform the `remindersLists` and `reminders` into a single collection
380+
of `Row` values:
381+
382+
```swift
383+
let rows = remindersLists.map { remindersList in
384+
Row(
385+
remindersList: remindersList,
386+
reminders: reminders.filter { reminder in
387+
reminder.remindersListID == remindersList.id
388+
}
389+
)
390+
}
391+
```
392+
393+
This can work, but it's incredibly inefficient, a lot of boilerplate, and prone to mistakes. And
394+
this is doing work that SQL actually excels at. In fact, the condition inside the `filter` looks
395+
suspiciously like a join constraint, which should give us a hint that what we are doing is not
396+
quite right.
397+
398+
Another way to do this is to use the `@Selection` macro described above
399+
(<doc:QueryCookbook#Custom-selections>), along with a ``JSONRepresentation`` of the collection
400+
of reminders you want to load for each list:
401+
402+
```struct
403+
@Selection
404+
struct Row {
405+
let remindersList: RemindersList
406+
@Column(as: JSONRepresentation<[Reminder]>.self)
407+
let reminders: [Reminder]
408+
}
409+
```
410+
411+
> Note: `Reminder` must conform to `Codable` to be able to use ``JSONRepresentation``.
412+
413+
This allows the query to serialize the associated rows into JSON, which are then deserialized into
414+
a `Row` type. To construct such a query you can use the
415+
``PrimaryKeyedTableDefinition/jsonGroupArray(order:filter:)`` property that is defined on the
416+
columns of [primary keyed tables](<doc:PrimaryKeyedTables>):
417+
418+
```swift
419+
RemindersList
420+
.join(Reminder.all) { $0.id.eq($1.remindersListID) }
421+
.select {
422+
Row.Columns(
423+
remindersList: $0,
424+
reminders: $1.jsonGroupArray()
425+
)
426+
}
427+
```
428+
429+
This allows you to fetch all of the data in a single SQLite query and decode the data into a
430+
collection of `Row` values. There is an extra cost associated with decoding the JSON object,
431+
but that cost may be smaller than executing multiple SQLite requests and transforming the data
432+
into `Row` manually, not to mention the additional code you need to write and maintain to process
433+
the data.

‎Sources/StructuredQueriesCore/QueryRepresentable/Codable+JSON.swift‎

Lines changed: 18 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ public struct JSONRepresentation<QueryOutput: Codable & Sendable>: QueryRepresen
2121

2222
public init(decoder: inout some QueryDecoder) throws {
2323
self.init(
24-
queryOutput: try JSONDecoder().decode(
24+
queryOutput: try jsonDecoder.decode(
2525
QueryOutput.self,
2626
from: Data(String(decoder: &decoder).utf8)
2727
)
@@ -32,7 +32,7 @@ public struct JSONRepresentation<QueryOutput: Codable & Sendable>: QueryRepresen
3232
extension JSONRepresentation: QueryBindable {
3333
public var queryBinding: QueryBinding {
3434
do {
35-
return try .text(String(decoding: JSONEncoder().encode(queryOutput), as: UTF8.self))
35+
return try .text(String(decoding: jsonEncoder.encode(queryOutput), as: UTF8.self))
3636
} catch {
3737
return .invalid(error)
3838
}
@@ -44,3 +44,19 @@ extension JSONRepresentation: SQLiteType {
4444
String.typeAffinity
4545
}
4646
}
47+
48+
private let jsonDecoder: JSONDecoder = {
49+
var decoder = JSONDecoder()
50+
decoder.dateDecodingStrategy = .custom {
51+
try $0.singleValueContainer().decode(String.self).iso8601
52+
}
53+
return decoder
54+
}()
55+
56+
private let jsonEncoder: JSONEncoder = {
57+
var encoder = JSONEncoder()
58+
#if DEBUG
59+
encoder.outputFormatting = [.prettyPrinted, .sortedKeys]
60+
#endif
61+
return encoder
62+
}()

‎Sources/StructuredQueriesCore/QueryRepresentable/Date+ISO8601.swift‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ extension Date.ISO8601Representation: QueryBindable {
3030

3131
extension Date.ISO8601Representation: QueryDecodable {
3232
public init(decoder: inout some QueryDecoder) throws {
33-
try self.init(queryOutput: String(decoder: &decoder).date)
33+
try self.init(queryOutput: String(decoder: &decoder).iso8601)
3434
}
3535
}
3636

@@ -69,7 +69,7 @@ extension DateFormatter {
6969
}
7070

7171
extension String {
72-
fileprivate var date: Date {
72+
var iso8601: Date {
7373
get throws {
7474
if #available(iOS 15, macOS 12, tvOS 15, watchOS 8, *) {
7575
do {

‎Sources/StructuredQueriesCore/SQLite/JSONFunctions.swift‎

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
import Foundation
2+
import StructuredQueriesSupport
3+
14
extension QueryExpression {
25
/// Wraps this expression with the `json_array_length` function.
36
///
@@ -34,3 +37,109 @@ extension QueryExpression where QueryValue: Codable & Sendable {
3437
AggregateFunction("json_group_array", self, order: order, filter: filter)
3538
}
3639
}
40+
41+
extension PrimaryKeyedTableDefinition where QueryValue: Codable & Sendable {
42+
/// A JSON array representation of the aggregation of a table's columns.
43+
///
44+
/// Constructs a JSON array of JSON objects with a field for each column of the table. This can be
45+
/// useful for loading many associated values in a single query. For example, to query for every
46+
/// reminders list, along with the array of reminders it is associated with, one can define a
47+
/// custom `@Selection` for that data and query as follows:
48+
///
49+
/// @Row {
50+
/// @Column {
51+
/// ```swift
52+
/// @Selection struct Row {
53+
/// let remindersList: RemindersList
54+
/// @Column(as: JSONRepresentation<[Reminder]>.self)
55+
/// let reminders: [Reminder]
56+
/// }
57+
/// RemindersList
58+
/// .join(Reminder.all) { $0.id.eq($1.remindersListID) }
59+
/// .select {
60+
/// Row.Columns(
61+
/// remindersList: $0,
62+
/// reminders: $1.jsonGroupArray()
63+
/// )
64+
/// }
65+
/// ```
66+
/// }
67+
/// @Column {
68+
/// ```sql
69+
/// SELECT
70+
/// "remindersLists".…,
71+
/// iif(
72+
/// "reminders"."id" IS NULL,
73+
/// NULL,
74+
/// json_object(
75+
/// 'id', json_quote("id"),
76+
/// 'title', json_quote("title"),
77+
/// 'priority', json_quote("priority")
78+
/// )
79+
/// )
80+
/// FROM "remindersLists"
81+
/// JOIN "reminders"
82+
/// ON ("remindersLists"."id" = "reminders"."remindersListID")
83+
/// ```
84+
/// }
85+
/// }
86+
///
87+
/// - Parameters:
88+
/// - order: An `ORDER BY` clause to apply to the aggregation.
89+
/// - filter: A `FILTER` clause to apply to the aggregation.
90+
/// - Returns: A JSON array aggregate of this table.
91+
public func jsonGroupArray(
92+
order: (some QueryExpression)? = Bool?.none,
93+
filter: (some QueryExpression<Bool>)? = Bool?.none
94+
) -> some QueryExpression<JSONRepresentation<[QueryValue]>> {
95+
jsonObject.jsonGroupArray(order: order, filter: filter)
96+
}
97+
98+
private var jsonObject: some QueryExpression<QueryValue> {
99+
func open<TableColumn: TableColumnExpression>(_ column: TableColumn) -> QueryFragment {
100+
typealias Value = TableColumn.QueryValue._Optionalized.Wrapped
101+
102+
func isJSONRepresentation<T: Codable & Sendable>(_: T.Type, isOptional: Bool = false) -> Bool
103+
{
104+
func isOptionalJSONRepresentation<U: _OptionalProtocol>(_: U.Type) -> Bool {
105+
if let codableType = U.Wrapped.self as? any (Codable & Sendable).Type {
106+
return isJSONRepresentation(codableType, isOptional: true)
107+
} else {
108+
return false
109+
}
110+
}
111+
if let optionalType = T.self as? any _OptionalProtocol.Type {
112+
return isOptionalJSONRepresentation(optionalType)
113+
} else if isOptional {
114+
return TableColumn.QueryValue.self == JSONRepresentation<T>?.self
115+
} else {
116+
return Value.self == JSONRepresentation<T>.self
117+
}
118+
}
119+
120+
if Value.self == Bool.self {
121+
return """
122+
\(quote: column.name, delimiter: .text), \
123+
json(CASE \(column) WHEN 0 THEN 'false' WHEN 1 THEN 'true' END)
124+
"""
125+
} else if Value.self == Date.UnixTimeRepresentation.self {
126+
return "\(quote: column.name, delimiter: .text), datetime(\(column), 'unixepoch')"
127+
} else if Value.self == Date.JulianDayRepresentation.self {
128+
return "\(quote: column.name, delimiter: .text), datetime(\(column), 'julianday')"
129+
} else if let codableType = TableColumn.QueryValue.QueryOutput.self
130+
as? any (Codable & Sendable).Type,
131+
isJSONRepresentation(codableType)
132+
{
133+
return "\(quote: column.name, delimiter: .text), json(\(column))"
134+
} else {
135+
return "\(quote: column.name, delimiter: .text), json_quote(\(column))"
136+
}
137+
}
138+
let fragment: QueryFragment = Self.allColumns
139+
.map { open($0) }
140+
.joined(separator: ", ")
141+
return SQLQueryExpression(
142+
"CASE WHEN \(primaryKey.isNot(nil)) THEN json_object(\(fragment)) END"
143+
)
144+
}
145+
}

0 commit comments

Comments
 (0)