Skip to content

Commit ead3d52

Browse files
cofincofin
authored
feat: improved type hinting (#26)
Improves the type hints of the each driver
1 parent a1c6ac8 commit ead3d52

File tree

18 files changed

+2292
-175
lines changed

18 files changed

+2292
-175
lines changed

‎README.md‎

Lines changed: 138 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,144 @@ SQLSpec is an experimental Python library designed to streamline and modernize y
2222

2323
SQLSpec is a work in progress. While it offers a solid foundation for modern SQL interactions, it does not yet include every feature you might find in a mature ORM or database toolkit. The focus is on building a robust, flexible core that can be extended over time.
2424

25+
## Examples
26+
27+
We've talked about what SQLSpec is not, so let's look at what it can do.
28+
29+
### Basic Example
30+
31+
### Multiple Database Engines
32+
33+
### Examples, Bells and Whistles
34+
35+
Let's take a look at a few alternate examples that leverage DuckDB and Litestar.
36+
37+
#### DuckDB LLM
38+
39+
This is a quick implementation using some of the built in Secret and Extension management features of SQLSpec's DuckDB integration.
40+
41+
It allows you to communicate with any compatible OpenAPI conversations endpoint (such as Ollama). This examples:
42+
43+
- auto installs the `open_prompt` DuckDB extensions
44+
- automatically creates the correct `open_prompt` comptaible secret required to use the extension
45+
46+
```py
47+
# /// script
48+
# dependencies = [
49+
# "sqlspec[duckdb,performance]",
50+
# ]
51+
# ///
52+
import os
53+
54+
from sqlspec import SQLSpec
55+
from sqlspec.adapters.duckdb import DuckDBConfig
56+
57+
sql = SQLSpec()
58+
etl_config = sql.add_config(
59+
DuckDBConfig(
60+
extensions=[{"name": "open_prompt"}],
61+
secrets=[
62+
{
63+
"secret_type": "open_prompt",
64+
"name": "open_prompt",
65+
"value": {
66+
"api_url": "http://127.0.0.1:11434/v1/chat/completions",
67+
"model_name": "gemma3:1b",
68+
"api_timeout": "120",
69+
},
70+
}
71+
],
72+
)
73+
)
74+
with sql.provide_session(etl_config) as session:
75+
result = session.select_one("SELECT generate_embedding('example text')")
76+
print(result)
77+
```
78+
79+
#### DuckDB Gemini Embeddings
80+
81+
In this example, we are again using DuckDB. However, we are going to use the built in to call the Google Gemini embeddings service directly from the database.
82+
83+
This example will
84+
85+
- auto installs the `http_client` and `vss` (vector similarity search) DuckDB extensions
86+
- when a connection is created, it ensures that the `generate_embeddings` macro exists in the DuckDB database.
87+
- Execute a simple query to call the Google API
88+
89+
```py
90+
# /// script
91+
# dependencies = [
92+
# "sqlspec[duckdb,performance]",
93+
# ]
94+
# ///
95+
import os
96+
97+
from sqlspec import SQLSpec
98+
from sqlspec.adapters.duckdb import DuckDBConfig
99+
100+
EMBEDDING_MODEL = "gemini-embedding-exp-03-07"
101+
GOOGLE_API_KEY = os.environ.get("GOOGLE_API_KEY")
102+
API_URL = (
103+
f"https://generativelanguage.googleapis.com/v1beta/models/{EMBEDDING_MODEL}:embedContent?key=${GOOGLE_API_KEY}"
104+
)
105+
106+
sql = SQLSpec()
107+
etl_config = sql.add_config(
108+
DuckDBConfig(
109+
extensions=[{"name": "vss"}, {"name": "http_client"}],
110+
on_connection_create=lambda connection: connection.execute(f"""
111+
CREATE IF NOT EXISTS MACRO generate_embedding(q) AS (
112+
WITH __request AS (
113+
SELECT http_post(
114+
'{API_URL}',
115+
headers => MAP {{
116+
'accept': 'application/json',
117+
}},
118+
params => MAP {{
119+
'model': 'models/{EMBEDDING_MODEL}',
120+
'parts': [{{ 'text': q }}],
121+
'taskType': 'SEMANTIC_SIMILARITY'
122+
}}
123+
) AS response
124+
)
125+
SELECT *
126+
FROM __request,
127+
);
128+
"""),
129+
)
130+
)
131+
```
132+
133+
#### Basic Litestar Integration
134+
135+
In this example we are going to demonstrate how to create a basic configuration that integrates into Litestar.
136+
137+
```py
138+
# /// script
139+
# dependencies = [
140+
# "sqlspec[aiosqlite]",
141+
# "litestar[standard]",
142+
# ]
143+
# ///
144+
145+
from aiosqlite import Connection
146+
from litestar import Litestar, get
147+
148+
from sqlspec.adapters.aiosqlite import AiosqliteConfig, AiosqliteDriver
149+
from sqlspec.extensions.litestar import SQLSpec
150+
151+
152+
@get("/")
153+
async def simple_sqlite(db_session: AiosqliteDriver) -> dict[str, str]:
154+
return await db_session.select_one("SELECT 'Hello, world!' AS greeting")
155+
156+
157+
sqlspec = SQLSpec(config=DatabaseConfig(
158+
config=[AiosqliteConfig(), commit_mode="autocommit")],
159+
)
160+
app = Litestar(route_handlers=[simple_sqlite], plugins=[sqlspec])
161+
```
162+
25163
## Inspiration and Future Direction
26164

27165
SQLSpec originally drew inspiration from features found in the `aiosql` library. This is a great library for working with and executed SQL stored in files. It's unclear how much of an overlap there will be between the two libraries, but it's possible that some features will be contributed back to `aiosql` where appropriate.

‎docs/PYPI_README.md‎

Lines changed: 138 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,144 @@ SQLSpec is an experimental Python library designed to streamline and modernize y
2222

2323
SQLSpec is a work in progress. While it offers a solid foundation for modern SQL interactions, it does not yet include every feature you might find in a mature ORM or database toolkit. The focus is on building a robust, flexible core that can be extended over time.
2424

25+
## Examples
26+
27+
We've talked about what SQLSpec is not, so let's look at what it can do.
28+
29+
### Basic Example
30+
31+
### Multiple Database Engines
32+
33+
### Examples, Bells and Whistles
34+
35+
Let's take a look at a few alternate examples that leverage DuckDB and Litestar.
36+
37+
#### DuckDB LLM
38+
39+
This is a quick implementation using some of the built in Secret and Extension management features of SQLSpec's DuckDB integration.
40+
41+
It allows you to communicate with any compatible OpenAPI conversations endpoint (such as Ollama). This examples:
42+
43+
- auto installs the `open_prompt` DuckDB extensions
44+
- automatically creates the correct `open_prompt` comptaible secret required to use the extension
45+
46+
```py
47+
# /// script
48+
# dependencies = [
49+
# "sqlspec[duckdb,performance]",
50+
# ]
51+
# ///
52+
import os
53+
54+
from sqlspec import SQLSpec
55+
from sqlspec.adapters.duckdb import DuckDBConfig
56+
57+
sql = SQLSpec()
58+
etl_config = sql.add_config(
59+
DuckDBConfig(
60+
extensions=[{"name": "open_prompt"}],
61+
secrets=[
62+
{
63+
"secret_type": "open_prompt",
64+
"name": "open_prompt",
65+
"value": {
66+
"api_url": "http://127.0.0.1:11434/v1/chat/completions",
67+
"model_name": "gemma3:1b",
68+
"api_timeout": "120",
69+
},
70+
}
71+
],
72+
)
73+
)
74+
with sql.provide_session(etl_config) as session:
75+
result = session.select_one("SELECT generate_embedding('example text')")
76+
print(result)
77+
```
78+
79+
#### DuckDB Gemini Embeddings
80+
81+
In this example, we are again using DuckDB. However, we are going to use the built in to call the Google Gemini embeddings service directly from the database.
82+
83+
This example will
84+
85+
- auto installs the `http_client` and `vss` (vector similarity search) DuckDB extensions
86+
- when a connection is created, it ensures that the `generate_embeddings` macro exists in the DuckDB database.
87+
- Execute a simple query to call the Google API
88+
89+
```py
90+
# /// script
91+
# dependencies = [
92+
# "sqlspec[duckdb,performance]",
93+
# ]
94+
# ///
95+
import os
96+
97+
from sqlspec import SQLSpec
98+
from sqlspec.adapters.duckdb import DuckDBConfig
99+
100+
EMBEDDING_MODEL = "gemini-embedding-exp-03-07"
101+
GOOGLE_API_KEY = os.environ.get("GOOGLE_API_KEY")
102+
API_URL = (
103+
f"https://generativelanguage.googleapis.com/v1beta/models/{EMBEDDING_MODEL}:embedContent?key=${GOOGLE_API_KEY}"
104+
)
105+
106+
sql = SQLSpec()
107+
etl_config = sql.add_config(
108+
DuckDBConfig(
109+
extensions=[{"name": "vss"}, {"name": "http_client"}],
110+
on_connection_create=lambda connection: connection.execute(f"""
111+
CREATE IF NOT EXISTS MACRO generate_embedding(q) AS (
112+
WITH __request AS (
113+
SELECT http_post(
114+
'{API_URL}',
115+
headers => MAP {{
116+
'accept': 'application/json',
117+
}},
118+
params => MAP {{
119+
'model': 'models/{EMBEDDING_MODEL}',
120+
'parts': [{{ 'text': q }}],
121+
'taskType': 'SEMANTIC_SIMILARITY'
122+
}}
123+
) AS response
124+
)
125+
SELECT *
126+
FROM __request,
127+
);
128+
"""),
129+
)
130+
)
131+
```
132+
133+
#### Basic Litestar Integration
134+
135+
In this example we are going to demonstrate how to create a basic configuration that integrates into Litestar.
136+
137+
```py
138+
# /// script
139+
# dependencies = [
140+
# "sqlspec[aiosqlite]",
141+
# "litestar[standard]",
142+
# ]
143+
# ///
144+
145+
from aiosqlite import Connection
146+
from litestar import Litestar, get
147+
148+
from sqlspec.adapters.aiosqlite import AiosqliteConfig, AiosqliteDriver
149+
from sqlspec.extensions.litestar import SQLSpec
150+
151+
152+
@get("/")
153+
async def simple_sqlite(db_session: AiosqliteDriver) -> dict[str, str]:
154+
return await db_session.select_one("SELECT 'Hello, world!' AS greeting")
155+
156+
157+
sqlspec = SQLSpec(config=DatabaseConfig(
158+
config=[AiosqliteConfig(), commit_mode="autocommit")],
159+
)
160+
app = Litestar(route_handlers=[simple_sqlite], plugins=[sqlspec])
161+
```
162+
25163
## Inspiration and Future Direction
26164

27165
SQLSpec originally drew inspiration from features found in the `aiosql` library. This is a great library for working with and executed SQL stored in files. It's unclear how much of an overlap there will be between the two libraries, but it's possible that some features will be contributed back to `aiosql` where appropriate.

‎docs/examples/litestar_duckllm.py‎

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -13,11 +13,11 @@
1313
# ]
1414
# ///
1515

16-
from duckdb import DuckDBPyConnection
1716
from litestar import Litestar, post
1817
from msgspec import Struct
1918

20-
from sqlspec.adapters.duckdb import DuckDB
19+
from sqlspec.adapters.duckdb import DuckDBConfig
20+
from sqlspec.adapters.duckdb.driver import DuckDBDriver
2121
from sqlspec.extensions.litestar import SQLSpec
2222

2323

@@ -26,13 +26,12 @@ class ChatMessage(Struct):
2626

2727

2828
@post("/chat", sync_to_thread=True)
29-
def duckllm_chat(db_connection: DuckDBPyConnection, data: ChatMessage) -> ChatMessage:
30-
result = db_connection.execute("SELECT open_prompt(?)", (data.message,)).fetchall()
31-
return ChatMessage(message=result[0][0])
29+
def duckllm_chat(db_session: DuckDBDriver, data: ChatMessage) -> ChatMessage:
30+
return db_session.select_one("SELECT open_prompt(?)", data.message, schema_type=ChatMessage)
3231

3332

3433
sqlspec = SQLSpec(
35-
config=DuckDB(
34+
config=DuckDBConfig(
3635
extensions=[{"name": "open_prompt"}],
3736
secrets=[
3837
{

‎docs/examples/litestar_multi_db.py‎

Lines changed: 34 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,34 +1,56 @@
1-
from aiosqlite import Connection
2-
from duckdb import DuckDBPyConnection
1+
"""Litestar Multi DB
2+
3+
This example demonstrates how to use multiple databases in a Litestar application.
4+
5+
The example uses the `SQLSpec` extension to create a connection to a SQLite (via `aiosqlite`) and DuckDB database.
6+
7+
The DuckDB database also demonstrates how to use the plugin loader and `secrets` configuration manager built into SQLSpec.
8+
"""
9+
# /// script
10+
# dependencies = [
11+
# "sqlspec[aiosqlite,duckdb]",
12+
# "litestar[standard]",
13+
# ]
14+
# ///
15+
316
from litestar import Litestar, get
417

5-
from sqlspec.adapters.aiosqlite import Aiosqlite
6-
from sqlspec.adapters.duckdb import DuckDB
18+
from sqlspec.adapters.aiosqlite import AiosqliteConfig, AiosqliteDriver
19+
from sqlspec.adapters.duckdb import DuckDBConfig, DuckDBDriver
720
from sqlspec.extensions.litestar import DatabaseConfig, SQLSpec
821

922

1023
@get("/test", sync_to_thread=True)
11-
def simple_select(etl_connection: DuckDBPyConnection) -> dict[str, str]:
12-
result = etl_connection.execute("SELECT 'Hello, world!' AS greeting").fetchall()
13-
return {"greeting": result[0][0]}
24+
def simple_select(etl_session: DuckDBDriver) -> dict[str, str]:
25+
result = etl_session.select_one("SELECT 'Hello, ETL world!' AS greeting")
26+
return {"greeting": result["greeting"]}
1427

1528

1629
@get("/")
17-
async def simple_sqlite(db_connection: Connection) -> dict[str, str]:
18-
result = await db_connection.execute_fetchall("SELECT 'Hello, world!' AS greeting")
19-
return {"greeting": result[0][0]} # type: ignore
30+
async def simple_sqlite(db_session: AiosqliteDriver) -> dict[str, str]:
31+
return await db_session.select_one("SELECT 'Hello, world!' AS greeting")
2032

2133

2234
sqlspec = SQLSpec(
2335
config=[
24-
DatabaseConfig(config=Aiosqlite(), commit_mode="autocommit"),
36+
DatabaseConfig(config=AiosqliteConfig(), commit_mode="autocommit"),
2537
DatabaseConfig(
26-
config=DuckDB(
38+
config=DuckDBConfig(
2739
extensions=[{"name": "vss", "force_install": True}],
2840
secrets=[{"secret_type": "s3", "name": "s3_secret", "value": {"key_id": "abcd"}}],
2941
),
3042
connection_key="etl_connection",
43+
session_key="etl_session",
3144
),
3245
],
3346
)
3447
app = Litestar(route_handlers=[simple_sqlite, simple_select], plugins=[sqlspec])
48+
49+
if __name__ == "__main__":
50+
import os
51+
52+
from litestar.cli import litestar_group
53+
54+
os.environ["LITESTAR_APP"] = "docs.examples.litestar_multi_db:app"
55+
56+
litestar_group()

0 commit comments

Comments
 (0)