Skip to content

Commit 27532c8

Browse files
authored
feat(config): simplify add_config return type (#278)
- Update the configuration registry to use instance IDs as keys, allowing multiple configurations of the same adapter type. - Modify the `add_config` method to return the configuration instance directly and ensure all relevant methods accept configuration instances. - Introduce validation for unregistered configurations and add a new method to retrieve configurations by type. - Remove deprecated methods and update tests accordingly.
1 parent 0ffb0d4 commit 27532c8

21 files changed

+676
-311
lines changed

AGENTS.md

Lines changed: 72 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,7 @@ make docs # Build Sphinx documentation
7171
### Adapters Structure
7272

7373
Each adapter in `sqlspec/adapters/` follows this structure:
74+
7475
- `config.py` - Configuration classes with pool settings
7576
- `driver.py` - Query execution implementation (sync/async)
7677
- `_types.py` - Adapter-specific type definitions
@@ -88,15 +89,14 @@ Supported adapters: adbc, aiosqlite, asyncmy, asyncpg, bigquery, duckdb, oracled
8889
### Database Connection Flow
8990

9091
```python
91-
# 1. Create configuration
92-
config = AsyncpgConfig(pool_config={"dsn": "postgresql://..."})
92+
# 1. Create SQLSpec manager
93+
manager = SQLSpec()
9394

94-
# 2. Register with SQLSpec
95-
sql = SQLSpec()
96-
sql.add_config(config)
95+
# 2. Create and register configuration (returns same instance)
96+
config = manager.add_config(AsyncpgConfig(pool_config={"dsn": "postgresql://..."}))
9797

98-
# 3. Get session via context manager
99-
async with sql.provide_session(config) as session:
98+
# 3. Get session via context manager (using config instance)
99+
async with manager.provide_session(config) as session:
100100
result = await session.execute("SELECT * FROM users")
101101
```
102102

@@ -133,6 +133,7 @@ async with sql.provide_session(config) as session:
133133
### Mypyc Compatibility
134134

135135
For classes in `sqlspec/core/` and `sqlspec/driver/`:
136+
136137
- Use `__slots__` for data-holding classes
137138
- Implement explicit `__init__`, `__repr__`, `__eq__`, `__hash__`
138139
- Avoid dataclasses in performance-critical paths
@@ -257,18 +258,72 @@ def database_to_bytes(value: Any) -> bytes | None:
257258
```
258259

259260
**Use this pattern when**:
261+
260262
- Database client library requires specific encoding for binary data
261263
- Need transparent conversion between Python bytes and database format
262264
- Want to centralize encoding/decoding logic for reuse
263265

264266
**Key principles**:
267+
265268
- Centralize conversion functions in `_type_handlers.py`
266269
- Handle None/NULL values explicitly
267270
- Support both raw and encoded formats on read (graceful handling)
268271
- Use in `coerce_params_for_database()` and type converter
269272

270273
**Example**: Spanner requires base64-encoded bytes for `param_types.BYTES` parameters, but you work with raw Python bytes in application code.
271274

275+
### Instance-Based Config Registry Pattern
276+
277+
SQLSpec uses config instances as handles for database connections. The registry keys by `id(config)` instead of `type(config)` to support multiple databases of the same adapter type.
278+
279+
```python
280+
from sqlspec import SQLSpec
281+
from sqlspec.adapters.asyncpg import AsyncpgConfig
282+
283+
manager = SQLSpec()
284+
285+
# Config instance IS the handle - add_config returns same instance
286+
main_db = manager.add_config(AsyncpgConfig(pool_config={"dsn": "postgresql://main/..."}))
287+
analytics_db = manager.add_config(AsyncpgConfig(pool_config={"dsn": "postgresql://analytics/..."}))
288+
289+
# Type checker knows: AsyncpgConfig → AsyncContextManager[AsyncpgDriver]
290+
async with manager.provide_session(main_db) as driver:
291+
await driver.execute("SELECT 1")
292+
293+
# Different connection pool! Works correctly now.
294+
async with manager.provide_session(analytics_db) as driver:
295+
await driver.execute("SELECT 1")
296+
```
297+
298+
**Use this pattern when**:
299+
300+
- Managing multiple databases of the same adapter type (e.g., main + analytics PostgreSQL)
301+
- Integrating with DI frameworks (config instance is the dependency)
302+
- Need type-safe session handles without `# type: ignore`
303+
304+
**Key principles**:
305+
306+
- Registry uses `id(config)` as key (not `type(config)`)
307+
- Multiple configs of same adapter type are stored separately
308+
- `add_config` returns the same instance passed in
309+
- All methods require registered config instances
310+
- Unregistered configs raise `ValueError`
311+
312+
**DI framework integration**:
313+
314+
```python
315+
# Just pass the config instance - it's already correctly typed
316+
def get_main_db() -> AsyncpgConfig:
317+
return main_db
318+
319+
# DI provider knows this is async from the config type
320+
async def provide_db_session(db: AsyncpgConfig, manager: SQLSpec):
321+
async with manager.provide_session(db) as driver:
322+
yield driver
323+
```
324+
325+
**Reference implementation**: `sqlspec/base.py` (lines 58, 128-151, 435-581)
326+
272327
### Framework Extension Pattern
273328

274329
All extensions use `extension_config` in database config:
@@ -323,17 +378,20 @@ _register_with_sqlglot()
323378
```
324379

325380
**Use this pattern when**:
381+
326382
- Database syntax varies significantly across dialects
327383
- Standard SQLGlot expressions don't match any database's native syntax
328384
- Need operator syntax (e.g., `<->`) vs function calls (e.g., `DISTANCE()`)
329385

330386
**Key principles**:
387+
331388
- Override `.sql()` method for dialect detection
332389
- Register with SQLGlot's TRANSFORMS for nested expression support
333390
- Store metadata (like metric) as `exp.Identifier` in `arg_types` for runtime access
334391
- Provide generic fallback for unsupported dialects
335392

336393
**Example**: `VectorDistance` in `sqlspec/builder/_vector_expressions.py` generates:
394+
337395
- PostgreSQL: `embedding <-> '[0.1,0.2]'` (operator)
338396
- MySQL: `DISTANCE(embedding, STRING_TO_VECTOR('[0.1,0.2]'), 'EUCLIDEAN')` (function)
339397
- Oracle: `VECTOR_DISTANCE(embedding, TO_VECTOR('[0.1,0.2]'), EUCLIDEAN)` (function)
@@ -388,17 +446,20 @@ Dialect.classes["custom"] = CustomDialect
388446
```
389447

390448
**Create custom dialect when**:
449+
391450
- Database has unique DDL/DML syntax not in existing dialects
392451
- Need to parse and validate database-specific keywords
393452
- Need to generate database-specific SQL from AST
394453
- An existing dialect provides 80%+ compatibility to inherit from
395454

396455
**Do NOT create custom dialect if**:
456+
397457
- Only parameter style differences (use parameter profiles)
398458
- Only type conversion differences (use type converters)
399459
- Only connection management differences (use config/driver)
400460

401461
**Key principles**:
462+
402463
- **Inherit from closest dialect**: Spanner inherits BigQuery (both GoogleSQL)
403464
- **Minimal overrides**: Only override methods that need customization
404465
- **Store metadata in AST**: Use `expression.set(key, value)` for custom data
@@ -443,17 +504,20 @@ def command(config: str | None):
443504
```
444505

445506
**Benefits:**
507+
446508
- Click automatically handles precedence (CLI flag always overrides env var)
447509
- Help text automatically shows env var name
448510
- Support for multiple fallback env vars via `envvar=["VAR1", "VAR2"]`
449511
- Less code, fewer bugs
450512

451513
**For project file discovery (pyproject.toml, etc.):**
514+
452515
- Use custom logic as fallback after Click env var handling
453516
- Walk filesystem from cwd to find config files
454517
- Return `None` if not found to trigger helpful error message
455518

456519
**Multi-config support:**
520+
457521
- Split comma-separated values from CLI flag, env var, or pyproject.toml
458522
- Resolve each config path independently
459523
- Flatten results if callables return lists
@@ -531,6 +595,7 @@ if async_configs:
531595
```
532596

533597
**Key principles:**
598+
534599
- Sync configs must execute outside event loop (direct function calls)
535600
- Async configs must execute inside event loop (via `run_()`)
536601
- Multi-config operations should batch sync and async separately for efficiency

docs/examples/adapters/adbc_postgres_ingest.py

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -126,13 +126,12 @@ def main(
126126
destination.parent.mkdir(parents=True, exist_ok=True)
127127

128128
db_manager = SQLSpec()
129-
adbc_config = AdbcConfig(connection_config={"uri": uri})
130-
adbc_key = db_manager.add_config(adbc_config)
129+
adbc_config = db_manager.add_config(AdbcConfig(connection_config={"uri": uri}))
131130

132-
_render_capabilities(console, db_manager.get_config(adbc_key))
131+
_render_capabilities(console, adbc_config)
133132
telemetry_records: list[dict[str, Any]] = []
134133

135-
with db_manager.provide_session(adbc_key) as session:
134+
with db_manager.provide_session(adbc_config) as session:
136135
export_job = session.select_to_storage(
137136
source_sql, str(destination), format_hint=file_format, partitioner=partitioner
138137
)

docs/examples/frameworks/fastapi/aiosqlite_app.py

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,14 +7,13 @@
77

88
from docs.examples.shared.configs import aiosqlite_registry
99
from docs.examples.shared.data import ARTICLES, CREATE_ARTICLES
10-
from sqlspec.adapters.aiosqlite import AiosqliteConfig, AiosqliteDriver
10+
from sqlspec.adapters.aiosqlite import AiosqliteDriver
1111
from sqlspec.core import SQL
1212

1313
__all__ = ("get_session", "list_articles", "main", "on_startup", "seed_database")
1414

1515

16-
registry = aiosqlite_registry()
17-
config = registry.get_config(AiosqliteConfig)
16+
registry, config = aiosqlite_registry()
1817
app = FastAPI()
1918

2019

docs/examples/frameworks/fastapi/sqlite_app.py

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,14 +8,13 @@
88

99
from docs.examples.shared.configs import sqlite_registry
1010
from docs.examples.shared.data import ARTICLES, CREATE_ARTICLES
11-
from sqlspec.adapters.sqlite import SqliteConfig, SqliteDriver
11+
from sqlspec.adapters.sqlite import SqliteDriver
1212
from sqlspec.core import SQL
1313

1414
__all__ = ("get_session", "list_articles", "main", "on_startup", "seed_database")
1515

1616

17-
registry = sqlite_registry()
18-
config = registry.get_config(SqliteConfig)
17+
registry, config = sqlite_registry()
1918
app = FastAPI()
2019

2120

docs/examples/frameworks/flask/sqlite_app.py

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,14 +4,12 @@
44

55
from docs.examples.shared.configs import sqlite_registry
66
from docs.examples.shared.data import ARTICLES, CREATE_ARTICLES
7-
from sqlspec.adapters.sqlite import SqliteConfig
87
from sqlspec.core import SQL
98

109
__all__ = ("list_articles", "main", "seed_database")
1110

1211

13-
registry = sqlite_registry()
14-
config = registry.get_config(SqliteConfig)
12+
registry, config = sqlite_registry()
1513
app = Flask(__name__)
1614

1715

docs/examples/frameworks/litestar/aiosqlite_app.py

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,12 +7,11 @@
77

88
from docs.examples.shared.configs import aiosqlite_registry
99
from docs.examples.shared.data import ARTICLES, CREATE_ARTICLES
10-
from sqlspec.adapters.aiosqlite import AiosqliteConfig, AiosqliteDriver
10+
from sqlspec.adapters.aiosqlite import AiosqliteDriver
1111
from sqlspec.core import SQL
1212
from sqlspec.extensions.litestar import SQLSpecPlugin
1313

14-
registry = aiosqlite_registry()
15-
config = registry.get_config(AiosqliteConfig)
14+
registry, config = aiosqlite_registry()
1615
plugin = SQLSpecPlugin(sqlspec=registry)
1716

1817

docs/examples/frameworks/litestar/duckdb_app.py

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,12 +6,11 @@
66

77
from docs.examples.shared.configs import duckdb_registry
88
from docs.examples.shared.data import ARTICLES, CREATE_ARTICLES
9-
from sqlspec.adapters.duckdb import DuckDBConfig, DuckDBDriver
9+
from sqlspec.adapters.duckdb import DuckDBDriver
1010
from sqlspec.core import SQL
1111
from sqlspec.extensions.litestar import SQLSpecPlugin
1212

13-
registry = duckdb_registry()
14-
config = registry.get_config(DuckDBConfig)
13+
registry, config = duckdb_registry()
1514
plugin = SQLSpecPlugin(sqlspec=registry)
1615

1716

docs/examples/frameworks/litestar/sqlite_app.py

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,12 +6,11 @@
66

77
from docs.examples.shared.configs import sqlite_registry
88
from docs.examples.shared.data import ARTICLES, CREATE_ARTICLES
9-
from sqlspec.adapters.sqlite import SqliteConfig, SqliteDriver
9+
from sqlspec.adapters.sqlite import SqliteDriver
1010
from sqlspec.core import SQL
1111
from sqlspec.extensions.litestar import SQLSpecPlugin
1212

13-
registry = sqlite_registry()
14-
config = registry.get_config(SqliteConfig)
13+
registry, config = sqlite_registry()
1514
plugin = SQLSpecPlugin(sqlspec=registry)
1615

1716

docs/examples/frameworks/starlette/aiosqlite_app.py

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,14 +9,12 @@
99

1010
from docs.examples.shared.configs import aiosqlite_registry
1111
from docs.examples.shared.data import ARTICLES, CREATE_ARTICLES
12-
from sqlspec.adapters.aiosqlite import AiosqliteConfig
1312
from sqlspec.core import SQL
1413

1514
__all__ = ("list_articles", "main", "seed_database")
1615

1716

18-
registry = aiosqlite_registry()
19-
config = registry.get_config(AiosqliteConfig)
17+
registry, config = aiosqlite_registry()
2018

2119

2220
async def seed_database() -> None:

docs/examples/shared/configs.py

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -8,34 +8,34 @@
88
__all__ = ("aiosqlite_registry", "duckdb_registry", "sqlite_registry")
99

1010

11-
def sqlite_registry(bind_key: str = "docs_sqlite") -> "SQLSpec":
11+
def sqlite_registry(bind_key: str = "docs_sqlite") -> "tuple[SQLSpec, SqliteConfig]":
1212
"""Return a registry with a single SQLite configuration."""
1313
registry = SQLSpec()
14-
registry.add_config(SqliteConfig(bind_key=bind_key, pool_config={"database": ":memory:"}))
15-
return registry
14+
config = registry.add_config(SqliteConfig(bind_key=bind_key, pool_config={"database": ":memory:"}))
15+
return registry, config
1616

1717

18-
def aiosqlite_registry(bind_key: str = "docs_aiosqlite") -> "SQLSpec":
18+
def aiosqlite_registry(bind_key: str = "docs_aiosqlite") -> "tuple[SQLSpec, AiosqliteConfig]":
1919
"""Return a registry backed by an AioSQLite pool."""
2020
registry = SQLSpec()
21-
registry.add_config(
21+
config = registry.add_config(
2222
AiosqliteConfig(
2323
bind_key=bind_key,
2424
pool_config={"database": ":memory:"},
2525
extension_config={"litestar": {"commit_mode": "autocommit"}},
2626
)
2727
)
28-
return registry
28+
return registry, config
2929

3030

31-
def duckdb_registry(bind_key: str = "docs_duckdb") -> "SQLSpec":
31+
def duckdb_registry(bind_key: str = "docs_duckdb") -> "tuple[SQLSpec, DuckDBConfig]":
3232
"""Return a registry with a DuckDB in-memory database."""
3333
registry = SQLSpec()
34-
registry.add_config(
34+
config = registry.add_config(
3535
DuckDBConfig(
3636
bind_key=bind_key,
3737
pool_config={"database": ":memory:shared_docs"},
3838
extension_config={"litestar": {"commit_mode": "autocommit"}},
3939
)
4040
)
41-
return registry
41+
return registry, config

0 commit comments

Comments
 (0)