Skip to content

Commit 5edbb60

Browse files
committed
feat: add StatementStack tests and enhance pipeline execution across adapters
- Implement tests for StatementStack sequential execution and continue-on-error behavior in BigQuery, DuckDB, Oracle, Psqlpy, Psycopg, and SQLite adapters. - Introduce edge case tests for StatementStack execution in SQLite, ensuring proper handling of empty stacks and mixed operations. - Enhance OracleAsyncDriver with pipeline support tests, verifying native pipeline execution and error handling. - Add metrics tracking for stack execution to monitor performance and error rates. - Create a utility script to run pre-commit hooks without requiring PTY support.
1 parent 2b948f6 commit 5edbb60

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

58 files changed

+2852
-201
lines changed

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ target/
1919
.vscode/
2020
.cursor/
2121
.zed/
22+
.cache
2223
.coverage*
2324
# files
2425
**/*.so

.pre-commit-config.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@ repos:
4343
rev: "v1.0.1"
4444
hooks:
4545
- id: sphinx-lint
46+
args: ["--jobs", "1"]
4647
- repo: local
4748
hooks:
4849
- id: pypi-readme

AGENTS.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -185,6 +185,25 @@ SQLSpec is a type-safe SQL query mapper designed for minimal abstraction between
185185
- **Single-Pass Processing**: Parse once → transform once → validate once - SQL object is single source of truth
186186
- **Abstract Methods with Concrete Implementations**: Protocol defines abstract methods, base classes provide concrete sync/async implementations
187187

188+
### Query Stack Implementation Guidelines
189+
190+
- **Builder Discipline**
191+
- `StatementStack` and `StackOperation` are immutable (`__slots__`, tuple storage). Every push helper returns a new stack; never mutate `_operations` in place.
192+
- Validate inputs at push time (non-empty SQL, execute_many payloads, reject nested stacks) so drivers can assume well-formed operations.
193+
- **Adapter Responsibilities**
194+
- Add a single capability gate per adapter (e.g., Oracle pipeline version check, `psycopg.capabilities.has_pipeline()`), return `super().execute_stack()` immediately when unsupported.
195+
- Preserve `StackResult.raw_result` by building SQL/Arrow results via `create_sql_result()` / `create_arrow_result()` instead of copying row data.
196+
- Honor manual toggles via `driver_features={"stack_native_disabled": True}` and document the behavior in the adapter guide.
197+
- **Telemetry + Tracing**
198+
- Always wrap adapter overrides with `StackExecutionObserver(self, stack, continue_on_error, native_pipeline=bool)`.
199+
- Do **not** emit duplicate metrics; the observer already increments `stack.execute.*`, logs `stack.execute.start/complete/failed`, and publishes the `sqlspec.stack.execute` span.
200+
- **Error Handling**
201+
- Wrap driver exceptions in `StackExecutionError` with `operation_index`, summarized SQL (`describe_stack_statement()`), adapter name, and execution mode.
202+
- Continue-on-error stacks append `StackResult.from_error()` and keep executing. Fail-fast stacks roll back (if they started the transaction) before re-raising the wrapped error.
203+
- **Testing Expectations**
204+
- Add integration tests under `tests/integration/test_adapters/<adapter>/test_driver.py::test_*statement_stack*` that cover native path, sequential fallback, and continue-on-error.
205+
- Guard base behavior (empty stacks, large stacks, transaction boundaries) via `tests/integration/test_stack_edge_cases.py`.
206+
188207
### Driver Parameter Profile Registry
189208

190209
- All adapter parameter defaults live in `DriverParameterProfile` entries inside `sqlspec/core/parameters.py`.

docs/changelog.rst

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,14 @@ SQLSpec Changelog
1010
Recent Updates
1111
==============
1212

13+
Query Stack Documentation Suite
14+
--------------------------------
15+
16+
- Expanded the :doc:`/reference/query-stack` API reference (``StatementStack``, ``StackResult``, driver hooks, and ``StackExecutionError``) with the high-level workflow, execution modes, telemetry, and troubleshooting tips.
17+
- Added :doc:`/examples/query_stack_example` that runs the same stack against SQLite and AioSQLite.
18+
- Captured the detailed architecture and performance guidance inside the internal specs workspace for future agent runs.
19+
- Updated every adapter reference with a **Query Stack Support** section so behavior is documented per database.
20+
1321
Migration Convenience Methods on Config Classes
1422
------------------------------------------------
1523

docs/examples/index.rst

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -91,6 +91,8 @@ Patterns
9191
- Routing requests to dedicated SQLite configs per tenant slug.
9292
* - ``patterns/configs/multi_adapter_registry.py``
9393
- Register multiple adapters on a single SQLSpec registry.
94+
* - ``query_stack_example.py``
95+
- Immutable StatementStack workflow executed against SQLite and AioSQLite drivers.
9496

9597
Loaders
9698
-------
@@ -142,4 +144,5 @@ Shared Utilities
142144
frameworks/starlette/aiosqlite_app
143145
frameworks/flask/sqlite_app
144146
patterns/configs/multi_adapter_registry
147+
query_stack_example
145148
README
Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
import asyncio
2+
3+
from sqlspec import SQLSpec
4+
from sqlspec.adapters.aiosqlite import AiosqliteConfig
5+
from sqlspec.adapters.sqlite import SqliteConfig
6+
from sqlspec.core import StatementStack
7+
8+
9+
def build_stack(user_id: int, action: str) -> "StatementStack":
10+
stack = (
11+
StatementStack()
12+
.push_execute(
13+
"INSERT INTO audit_log (user_id, action) VALUES (:user_id, :action)", {"user_id": user_id, "action": action}
14+
)
15+
.push_execute(
16+
"UPDATE users SET last_action = :action WHERE id = :user_id", {"action": action, "user_id": user_id}
17+
)
18+
.push_execute("SELECT role FROM user_roles WHERE user_id = :user_id ORDER BY role", {"user_id": user_id})
19+
)
20+
return stack
21+
22+
23+
def run_sync_example() -> None:
24+
sql = SQLSpec()
25+
config = SqliteConfig(pool_config={"database": ":memory:"})
26+
registry = sql.add_config(config)
27+
28+
with sql.provide_session(registry) as session:
29+
session.execute_script(
30+
"""
31+
CREATE TABLE IF NOT EXISTS users (id INTEGER PRIMARY KEY, last_action TEXT);
32+
CREATE TABLE IF NOT EXISTS audit_log (
33+
id INTEGER PRIMARY KEY AUTOINCREMENT,
34+
user_id INTEGER NOT NULL,
35+
action TEXT NOT NULL
36+
);
37+
CREATE TABLE IF NOT EXISTS user_roles (
38+
user_id INTEGER NOT NULL,
39+
role TEXT NOT NULL
40+
);
41+
INSERT INTO users (id, last_action) VALUES (1, 'start');
42+
INSERT INTO user_roles (user_id, role) VALUES (1, 'admin'), (1, 'editor');
43+
"""
44+
)
45+
46+
stack = build_stack(user_id=1, action="sync-login")
47+
results = session.execute_stack(stack)
48+
49+
audit_insert, user_update, role_select = results
50+
print("[sync] rows inserted:", audit_insert.rowcount)
51+
print("[sync] rows updated:", user_update.rowcount)
52+
if role_select.raw_result is not None:
53+
print("[sync] roles:", [row["role"] for row in role_select.raw_result.data])
54+
55+
56+
def run_async_example() -> None:
57+
async def _inner() -> None:
58+
sql = SQLSpec()
59+
config = AiosqliteConfig(pool_config={"database": ":memory:"})
60+
registry = sql.add_config(config)
61+
62+
async with sql.provide_session(registry) as session:
63+
await session.execute_script(
64+
"""
65+
CREATE TABLE IF NOT EXISTS users (id INTEGER PRIMARY KEY, last_action TEXT);
66+
CREATE TABLE IF NOT EXISTS audit_log (
67+
id INTEGER PRIMARY KEY AUTOINCREMENT,
68+
user_id INTEGER NOT NULL,
69+
action TEXT NOT NULL
70+
);
71+
CREATE TABLE IF NOT EXISTS user_roles (
72+
user_id INTEGER NOT NULL,
73+
role TEXT NOT NULL
74+
);
75+
INSERT INTO users (id, last_action) VALUES (2, 'start');
76+
INSERT INTO user_roles (user_id, role) VALUES (2, 'viewer');
77+
"""
78+
)
79+
80+
stack = build_stack(user_id=2, action="async-login")
81+
results = await session.execute_stack(stack, continue_on_error=False)
82+
83+
audit_insert, user_update, role_select = results
84+
print("[async] rows inserted:", audit_insert.rowcount)
85+
print("[async] rows updated:", user_update.rowcount)
86+
if role_select.raw_result is not None:
87+
print("[async] roles:", [row["role"] for row in role_select.raw_result.data])
88+
89+
asyncio.run(_inner())
90+
91+
92+
def main() -> None:
93+
run_sync_example()
94+
run_async_example()
95+
96+
97+
if __name__ == "__main__":
98+
main()
Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
====================
2+
Query Stack Example
3+
====================
4+
5+
This example builds an immutable ``StatementStack`` and executes it against both the synchronous SQLite adapter and the asynchronous AioSQLite adapter. Each stack:
6+
7+
1. Inserts an audit log row
8+
2. Updates the user's last action
9+
3. Fetches the user's roles
10+
11+
.. literalinclude:: query_stack_example.py
12+
:language: python
13+
:caption: ``docs/examples/query_stack_example.py``
14+
:linenos:
15+
16+
Run the script:
17+
18+
.. code-block:: console
19+
20+
uv run python docs/examples/query_stack_example.py
21+
22+
Expected output shows inserted/updated row counts plus the projected role list for each adapter.

docs/guides/README.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,11 @@ Optimization guides for SQLSpec:
3636

3737
- [**SQLglot Guide**](performance/sqlglot.md) - SQL parsing, transformation, and optimization with SQLglot
3838
- [**MyPyC Guide**](performance/mypyc.md) - Compilation strategies for high-performance Python code
39+
- [**Batch Execution**](performance/batch-execution.md) - Guidance for Query Stack vs. ``execute_many`` across adapters
40+
41+
## Features
42+
43+
- [**Query Stack Guide**](features/query-stack.md) - Multi-statement execution, execution modes, telemetry, and troubleshooting
3944

4045
## Migrations
4146

@@ -55,6 +60,7 @@ Core architecture and design patterns:
5560

5661
- [**Architecture Guide**](architecture/architecture.md) - SQLSpec architecture overview
5762
- [**Data Flow Guide**](architecture/data-flow.md) - How data flows through SQLSpec
63+
- [**Architecture Patterns**](architecture/patterns.md) - Immutable stack builder, native vs. sequential branching, and telemetry requirements
5864

5965
## Extensions
6066

docs/guides/adapters/adbc.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,12 @@ This guide provides specific instructions for the `adbc` adapter.
1717
- **JSON Strategy:** `helper` (shared serializers wrap dict/list/tuple values)
1818
- **Extras:** `type_coercion_overrides` ensure Arrow arrays map to Python lists; PostgreSQL dialects attach a NULL-handling AST transformer
1919

20+
## Query Stack Support
21+
22+
- Each ADBC backend falls back to SQLSpec's sequential stack executor. There is no driver-agnostic pipeline API today, so stacks simply reuse the same cursor management that individual `execute()` calls use, wrapped in a transaction when the backend supports it (e.g., PostgreSQL) and as independent statements when it does not (e.g., SQLite, DuckDB).
23+
- Continue-on-error mode is supported on every backend. Successful statements commit as they finish, while failures populate `StackResult.error` for downstream inspection.
24+
- Telemetry spans (`sqlspec.stack.execute`) and `StackExecutionMetrics` counters emit for all stacks, enabling observability parity with adapters that do have native optimizations.
25+
2026
## Best Practices
2127

2228
- **Arrow-Native:** The primary benefit of ADBC is its direct integration with Apache Arrow. Use it when you need to move large amounts of data efficiently between the database and data science tools like Pandas or Polars.

docs/guides/adapters/aiosqlite.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,11 @@ This guide provides specific instructions for the `aiosqlite` adapter.
1717
- **JSON Strategy:** `helper` (shared serializer handles dict/list/tuple inputs)
1818
- **Extras:** None (profile applies bool→int and ISO datetime coercions automatically)
1919

20+
## Query Stack Support
21+
22+
- `StatementStack` executions always use the sequential fallback – SQLite has no notion of pipelined requests – so each operation runs one after another on the same connection. When `continue_on_error=False`, SQLSpec opens a transaction (if one is not already in progress) so the entire stack commits or rolls back together. With `continue_on_error=True`, statements are committed individually after each success.
23+
- Because pooled in-memory connections share state, prefer per-test temporary database files when running stacks under pytest-xdist (see `tests/integration/test_adapters/test_aiosqlite/test_driver.py::test_aiosqlite_statement_stack_*` for the reference pattern).
24+
2025
## Best Practices
2126

2227
- **Async Only:** This is an asynchronous driver for SQLite. Use it in `asyncio` applications.

0 commit comments

Comments
 (0)