You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
**mquire queries require reconstructing kernel data structures from virtual memory by dereferencing pointers using embedded type information and debug symbols. This processing can be expensive, so use query optimization techniques to improve performance dramatically.**
165
+
166
+
### Materialization with `AS MATERIALIZED`
167
+
168
+
Use the `AS MATERIALIZED` hint to cache table results when tables are used in JOINs or accessed multiple times.
169
+
170
+
**When to materialize:**
171
+
- Tables that are expensive to generate (e.g., `tasks` requires walking linked lists of process structures, dereferencing multiple pointers per process)
172
+
- Tables used in JOINs (accessed multiple times during query execution)
173
+
- Tables referenced multiple times in the same query
174
+
175
+
**Example:**
176
+
177
+
```sql
178
+
WITH
179
+
tasks_mat AS MATERIALIZED (
180
+
SELECT*FROM tasks
181
+
),
182
+
183
+
network_connections_mat AS MATERIALIZED (
184
+
SELECT*FROM network_connections
185
+
),
186
+
187
+
task_open_files_mat AS MATERIALIZED (
188
+
SELECT*FROM task_open_files
189
+
)
190
+
191
+
SELECT
192
+
t.pid,
193
+
t.comm,
194
+
nc.local_address,
195
+
nc.local_port,
196
+
nc.remote_address,
197
+
nc.remote_port,
198
+
nc.state,
199
+
nc.protocol
200
+
FROM network_connections_mat nc
201
+
JOIN task_open_files_mat tof ONnc.inode=tof.inode
202
+
JOIN tasks_mat t ONtof.pid=t.pid;
203
+
```
204
+
205
+
**Performance impact:** Materialization can provide significant speedup for queries with JOINs (typically 2-5x faster)
206
+
207
+
**Example benchmark results:**
208
+
209
+
*Test performed on an Ubuntu 24.04 snapshot (kernel 6.8.0-63), 351 processes, 50 connections, 2142 open files. Performance will vary based on snapshot size, kernel version, and hardware.*
210
+
211
+
| Method | Real Time | User Time | Speedup |
212
+
|--------|-----------|-----------|---------|
213
+
| WITHOUT materialization | 12.067s | 16.373s | baseline |
214
+
| WITH materialization | 3.171s | 8.786s |**3.8x faster**|
215
+
216
+
### JOIN Order Optimization
217
+
218
+
**Start with the smallest table and JOIN toward larger tables** to minimize rows processed early in the query pipeline.
219
+
220
+
**Typical table sizes:**
221
+
-`network_connections`: smallest - only processes with network activity
222
+
-`tasks`: medium - all processes
223
+
-`task_open_files`: largest - all open file descriptors
224
+
225
+
**Optimal order:**
226
+
```sql
227
+
-- Good: Starts with smallest table
228
+
FROM network_connections_mat nc -- ~50 rows
229
+
JOIN task_open_files_mat tof ON ... -- ~2000 rows
230
+
JOIN tasks_mat t ON ... -- ~350 rows
231
+
232
+
-- Less optimal: Starts with larger table
233
+
FROM task_open_files_mat tof -- ~2000 rows
234
+
JOIN network_connections_mat nc ON ... -- ~50 rows
235
+
JOIN tasks_mat t ON ... -- ~350 rows
236
+
```
237
+
238
+
### Understanding Query Execution
239
+
240
+
Use `EXPLAIN QUERY PLAN` to see how SQLite executes your query:
241
+
242
+
```sql
243
+
EXPLAIN QUERY PLAN
244
+
SELECT ...
245
+
FROM network_connections_mat nc
246
+
JOIN task_open_files_mat tof ONnc.inode=tof.inode
247
+
JOIN tasks_mat t ONtof.pid=t.pid;
248
+
```
249
+
250
+
Look for:
251
+
-**BLOOM FILTER**: SQLite's optimization for large JOINs
252
+
-**AUTOMATIC COVERING INDEX**: Temporary indexes created for lookups
253
+
-**SCAN**: Full table scan (expected for the driving table)
254
+
-**SEARCH**: Index-based lookup (efficient)
255
+
256
+
### Best Practices
257
+
258
+
1.**Always materialize expensive tables** used in JOINs
259
+
2.**Start with the smallest table** as your driving table
260
+
3.**Use multiline SQL** in scripts for readability
261
+
4.**Check query plans** with `EXPLAIN QUERY PLAN` for complex queries
262
+
5.**Avoid `SELECT *`** in production - specify only needed columns
Note: This query uses materialization for better performance (see [Query Optimization](#query-optimization) section).
322
+
219
323
```bash
220
324
$ mquire shell ubuntu2404_6.14.0-37-generic.lime
221
-
mquire> SELECT t.pid, t.comm, nc.local_address, nc.local_port, nc.remote_address, nc.remote_port, nc.state, nc.protocol FROM network_connections nc JOIN task_open_files tof ON nc.inode = tof.inode JOIN tasks t ON tof.pid = t.pid;
325
+
mquire>WITH tasks_mat AS MATERIALIZED (SELECT * FROM tasks), network_connections_mat AS MATERIALIZED (SELECT * FROM network_connections), task_open_files_mat AS MATERIALIZED (SELECT * FROM task_open_files) SELECT t.pid, t.comm, nc.local_address, nc.local_port, nc.remote_address, nc.remote_port, nc.state, nc.protocol FROM network_connections_mat nc JOIN task_open_files_mat tof ON nc.inode = tof.inode JOIN tasks_mat t ON tof.pid = t.pid;
0 commit comments