|
2 | 2 | <p align="center">
|
3 | 3 | <img src="https://raw.githubusercontent.com/litestar-org/branding/refs/heads/main/assets/Branding%20-%20SVG%20-%20Transparent/AA%20-%20Banner%20-%20Inline%20-%20Light.svg" alt="Litestar Logo - Light" width="100%" height="auto" />
|
4 | 4 | </p>
|
| 5 | +<div align="center"> |
| 6 | +<!-- markdownlint-restore --> |
| 7 | + |
| 8 | +<!-- prettier-ignore-start --> |
| 9 | + |
| 10 | +| Project | | Status | |
| 11 | +|-----------|:----|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| |
| 12 | +| CI/CD | | [](https://github.com/litestar-org/advanced-alchemy/actions/workflows/publish.yml) [](https://github.com/litestar-org/advanced-alchemy/actions/workflows/ci.yml) [](https://github.com/litestar-org/advanced-alchemy/actions/workflows/docs.yml) | |
| 13 | +| Quality | | [](https://codecov.io/github/litestar-org/advanced-alchemy) [](https://sonarcloud.io/summary/new_code?id=litestar-org_advanced-alchemy) [](https://sonarcloud.io/summary/new_code?id=litestar-org_advanced-alchemy) [](https://sonarcloud.io/summary/new_code?id=litestar-org_advanced-alchemy) [](https://sonarcloud.io/summary/new_code?id=litestar-org_advanced-alchemy) | |
| 14 | +| Package | | [](https://badge.fury.io/py/advanced-alchemy)   | |
| 15 | +| Community | | [](https://discord.gg/litestar) [](https://matrix.to/#/#litestar:matrix.org) | |
| 16 | +| Meta | | [](https://github.com/litestar-org/advanced-alchemy) [](https://github.com/python/mypy) [](https://spdx.org/licenses/) [](https://github.com/astral-sh/ruff) | |
| 17 | + |
| 18 | +</div> |
| 19 | + |
| 20 | +# Advanced Alchemy |
| 21 | + |
| 22 | +Check out the [project documentation][project-docs] 📚 for more information. |
| 23 | + |
| 24 | +## About |
| 25 | + |
| 26 | +A carefully crafted, thoroughly tested, optimized companion library for SQLAlchemy, |
| 27 | +offering: |
| 28 | + |
| 29 | +- Sync and async repositories, featuring common CRUD and highly optimized bulk operations |
| 30 | +- Integration with major web frameworks including Litestar, Starlette, FastAPI, Sanic |
| 31 | +- Custom-built alembic configuration and CLI with optional framework integration |
| 32 | +- Utility base classes with audit columns, primary keys and utility functions |
| 33 | +- Built in `File Object` data type for storing objects: |
| 34 | + - Unified interface for various storage backends ([`fsspec`](https://filesystem-spec.readthedocs.io/en/latest/) and [`obstore`](https://developmentseed.org/obstore/latest/)) |
| 35 | + - Optional lifecycle event hooks integrated with SQLAlchemy's event system to automatically save and delete files as records are inserted, updated, or deleted. |
| 36 | +- Optimized JSON types including a custom JSON type for Oracle |
| 37 | +- Integrated support for UUID6 and UUID7 using [`uuid-utils`](https://github.com/aminalaee/uuid-utils) (install with the `uuid` extra) |
| 38 | +- Integrated support for Nano ID using [`fastnanoid`](https://github.com/oliverlambson/fastnanoid) (install with the `nanoid` extra) |
| 39 | +- Custom encrypted text type with multiple backend support including [`pgcrypto`](https://www.postgresql.org/docs/current/pgcrypto.html) for PostgreSQL and the Fernet implementation from [`cryptography`](https://cryptography.io/en/latest/) for other databases |
| 40 | +- Custom password hashing type with multiple backend support including [`Argon2`](https://github.com/P-H-C/phc-winner-argon2), [`Passlib`](https://passlib.readthedocs.io/en/stable/), and [`Pwdlib`](https://pwdlib.readthedocs.io/en/stable/) with automatic salt generation |
| 41 | +- Pre-configured base classes with audit columns UUID or Big Integer primary keys and |
| 42 | + a [sentinel column](https://docs.sqlalchemy.org/en/20/core/connections.html#configuring-sentinel-columns). |
| 43 | +- Synchronous and asynchronous repositories featuring: |
| 44 | + - Common CRUD operations for SQLAlchemy models |
| 45 | + - Bulk inserts, updates, upserts, and deletes with dialect-specific enhancements |
| 46 | + - Integrated counts, pagination, sorting, filtering with `LIKE`, `IN`, and dates before and/or after. |
| 47 | +- Tested support for multiple database backends including: |
| 48 | + - SQLite via [aiosqlite](https://aiosqlite.omnilib.dev/en/stable/) or [sqlite](https://docs.python.org/3/library/sqlite3.html) |
| 49 | + - Postgres via [asyncpg](https://magicstack.github.io/asyncpg/current/) or [psycopg3 (async or sync)](https://www.psycopg.org/psycopg3/) |
| 50 | + - MySQL via [asyncmy](https://github.com/long2ice/asyncmy) |
| 51 | + - Oracle via [oracledb (async or sync)](https://oracle.github.io/python-oracledb/) (tested on 18c and 23c) |
| 52 | + - Google Spanner via [spanner-sqlalchemy](https://github.com/googleapis/python-spanner-sqlalchemy/) |
| 53 | + - DuckDB via [duckdb_engine](https://github.com/Mause/duckdb_engine) |
| 54 | + - Microsoft SQL Server via [pyodbc](https://github.com/mkleehammer/pyodbc) or [aioodbc](https://github.com/aio-libs/aioodbc) |
| 55 | + - CockroachDB via [sqlalchemy-cockroachdb (async or sync)](https://github.com/cockroachdb/sqlalchemy-cockroachdb) |
| 56 | +- ...and much more |
| 57 | + |
| 58 | +## Usage |
| 59 | + |
| 60 | +### Installation |
| 61 | + |
| 62 | +```shell |
| 63 | +pip install advanced-alchemy |
| 64 | +``` |
| 65 | + |
| 66 | +> [!IMPORTANT]\ |
| 67 | +> Check out [the installation guide][install-guide] in our official documentation! |
| 68 | +
|
| 69 | +### Repositories |
| 70 | + |
| 71 | +Advanced Alchemy includes a set of asynchronous and synchronous repository classes for easy CRUD |
| 72 | +operations on your SQLAlchemy models. |
| 73 | +<!-- markdownlint-disable --> |
| 74 | +<details> |
| 75 | +<summary>Click to expand the example</summary> |
| 76 | +<!-- markdownlint-restore --> |
| 77 | + |
| 78 | +```python |
| 79 | +from advanced_alchemy import base, repository, config |
| 80 | +from sqlalchemy import create_engine |
| 81 | +from sqlalchemy.orm import Mapped, sessionmaker |
| 82 | + |
| 83 | + |
| 84 | +class User(base.UUIDBase): |
| 85 | + # you can optionally override the generated table name by manually setting it. |
| 86 | + __tablename__ = "user_account" # type: ignore[assignment] |
| 87 | + email: Mapped[str] |
| 88 | + name: Mapped[str] |
| 89 | + |
| 90 | + |
| 91 | +class UserRepository(repository.SQLAlchemySyncRepository[User]): |
| 92 | + """User repository.""" |
| 93 | + |
| 94 | + model_type = User |
| 95 | + |
| 96 | + |
| 97 | +db = config.SQLAlchemySyncConfig(connection_string="duckdb:///:memory:", session_config=config.SyncSessionConfig(expire_on_commit=False)) |
| 98 | + |
| 99 | +# Initializes the database. |
| 100 | +with db.get_engine().begin() as conn: |
| 101 | + User.metadata.create_all(conn) |
| 102 | + |
| 103 | +with db.get_session() as db_session: |
| 104 | + repo = UserRepository(session=db_session) |
| 105 | + # 1) Create multiple users with `add_many` |
| 106 | + bulk_users = [ |
| 107 | + { "email": '[email protected]', 'name': 'Cody'}, |
| 108 | + { "email": '[email protected]', 'name': 'Janek'}, |
| 109 | + { "email": '[email protected]', 'name': 'Peter'}, |
| 110 | + { "email": '[email protected]', 'name': 'Jacob'} |
| 111 | + ] |
| 112 | + objs = repo.add_many([User(**raw_user) for raw_user in bulk_users]) |
| 113 | + db_session.commit() |
| 114 | + print(f"Created {len(objs)} new objects.") |
| 115 | + |
| 116 | + # 2) Select paginated data and total row count. Pass additional filters as kwargs |
| 117 | + created_objs, total_objs = repo.list_and_count(LimitOffset(limit=10, offset=0), name="Cody") |
| 118 | + print(f"Selected {len(created_objs)} records out of a total of {total_objs}.") |
| 119 | + |
| 120 | + # 3) Let's remove the batch of records selected. |
| 121 | + deleted_objs = repo.delete_many([new_obj.id for new_obj in created_objs]) |
| 122 | + print(f"Removed {len(deleted_objs)} records out of a total of {total_objs}.") |
| 123 | + |
| 124 | + # 4) Let's count the remaining rows |
| 125 | + remaining_count = repo.count() |
| 126 | + print(f"Found {remaining_count} remaining records after delete.") |
| 127 | +``` |
| 128 | + |
| 129 | +</details> |
| 130 | + |
| 131 | +For a full standalone example, see the sample [here][standalone-example] |
| 132 | + |
| 133 | +### Services |
| 134 | + |
| 135 | +Advanced Alchemy includes an additional service class to make working with a repository easier. |
| 136 | +This class is designed to accept data as a dictionary or SQLAlchemy model, |
| 137 | +and it will handle the type conversions for you. |
| 138 | +<!-- markdownlint-disable --> |
| 139 | +<details> |
| 140 | +<summary>Here's the same example from above but using a service to create the data:</summary> |
| 141 | +<!-- markdownlint-restore --> |
| 142 | + |
| 143 | +```python |
| 144 | +from advanced_alchemy import base, repository, filters, service, config |
| 145 | +from sqlalchemy import create_engine |
| 146 | +from sqlalchemy.orm import Mapped, sessionmaker |
| 147 | + |
| 148 | + |
| 149 | +class User(base.UUIDBase): |
| 150 | + # you can optionally override the generated table name by manually setting it. |
| 151 | + __tablename__ = "user_account" # type: ignore[assignment] |
| 152 | + email: Mapped[str] |
| 153 | + name: Mapped[str] |
| 154 | + |
| 155 | +class UserService(service.SQLAlchemySyncRepositoryService[User]): |
| 156 | + """User repository.""" |
| 157 | + class Repo(repository.SQLAlchemySyncRepository[User]): |
| 158 | + """User repository.""" |
| 159 | + |
| 160 | + model_type = User |
| 161 | + |
| 162 | + repository_type = Repo |
| 163 | + |
| 164 | +db = config.SQLAlchemySyncConfig(connection_string="duckdb:///:memory:", session_config=config.SyncSessionConfig(expire_on_commit=False)) |
| 165 | + |
| 166 | +# Initializes the database. |
| 167 | +with db.get_engine().begin() as conn: |
| 168 | + User.metadata.create_all(conn) |
| 169 | + |
| 170 | +with db.get_session() as db_session: |
| 171 | + service = UserService(session=db_session) |
| 172 | + # 1) Create multiple users with `add_many` |
| 173 | + objs = service.create_many([ |
| 174 | + { "email": '[email protected]', 'name': 'Cody'}, |
| 175 | + { "email": '[email protected]', 'name': 'Janek'}, |
| 176 | + { "email": '[email protected]', 'name': 'Peter'}, |
| 177 | + { "email": '[email protected]', 'name': 'Jacob'} |
| 178 | + ]) |
| 179 | + print(objs) |
| 180 | + print(f"Created {len(objs)} new objects.") |
| 181 | + |
| 182 | + # 2) Select paginated data and total row count. Pass additional filters as kwargs |
| 183 | + created_objs, total_objs = service.list_and_count(LimitOffset(limit=10, offset=0), name="Cody") |
| 184 | + print(f"Selected {len(created_objs)} records out of a total of {total_objs}.") |
| 185 | + |
| 186 | + # 3) Let's remove the batch of records selected. |
| 187 | + deleted_objs = service.delete_many([new_obj.id for new_obj in created_objs]) |
| 188 | + print(f"Removed {len(deleted_objs)} records out of a total of {total_objs}.") |
| 189 | + |
| 190 | + # 4) Let's count the remaining rows |
| 191 | + remaining_count = service.count() |
| 192 | + print(f"Found {remaining_count} remaining records after delete.") |
| 193 | +``` |
| 194 | + |
| 195 | +</details> |
| 196 | + |
| 197 | +### Web Frameworks |
| 198 | + |
| 199 | +Advanced Alchemy works with nearly all Python web frameworks. |
| 200 | +Several helpers for popular libraries are included, and additional PRs to support others are welcomed. |
| 201 | + |
| 202 | +#### Litestar |
| 203 | + |
| 204 | +Advanced Alchemy is the official SQLAlchemy integration for Litestar. |
| 205 | + |
| 206 | +In addition to installing with `pip install advanced-alchemy`, |
| 207 | +it can also be installed as a Litestar extra with `pip install litestar[sqlalchemy]`. |
| 208 | + |
| 209 | +<!-- markdownlint-disable --> |
| 210 | +<details> |
| 211 | +<summary>Litestar Example</summary> |
| 212 | +<!-- markdownlint-restore --> |
| 213 | + |
| 214 | +```python |
| 215 | +from litestar import Litestar |
| 216 | +from litestar.plugins.sqlalchemy import SQLAlchemyPlugin, SQLAlchemyAsyncConfig |
| 217 | +# alternately... |
| 218 | +# from advanced_alchemy.extensions.litestar import SQLAlchemyAsyncConfig, SQLAlchemyPlugin |
| 219 | + |
| 220 | +alchemy = SQLAlchemyPlugin( |
| 221 | + config=SQLAlchemyAsyncConfig(connection_string="sqlite+aiosqlite:///test.sqlite"), |
| 222 | +) |
| 223 | +app = Litestar(plugins=[alchemy]) |
| 224 | +``` |
| 225 | + |
| 226 | +</details> |
| 227 | + |
| 228 | +For a full Litestar example, check [here][litestar-example] |
| 229 | + |
| 230 | +#### Flask |
| 231 | + |
| 232 | +<!-- markdownlint-disable --> |
| 233 | +<details> |
| 234 | +<summary>Flask Example</summary> |
| 235 | +<!-- markdownlint-restore --> |
| 236 | + |
| 237 | +```python |
| 238 | +from flask import Flask |
| 239 | +from advanced_alchemy.extensions.flask import AdvancedAlchemy, SQLAlchemySyncConfig |
| 240 | + |
| 241 | +app = Flask(__name__) |
| 242 | +alchemy = AdvancedAlchemy( |
| 243 | + config=SQLAlchemySyncConfig(connection_string="duckdb:///:memory:"), app=app, |
| 244 | +) |
| 245 | +``` |
| 246 | + |
| 247 | +</details> |
| 248 | + |
| 249 | +For a full Flask example, see [here][flask-example] |
| 250 | + |
| 251 | +#### FastAPI |
| 252 | + |
| 253 | +<!-- markdownlint-disable --> |
| 254 | +<details> |
| 255 | +<summary>FastAPI Example</summary> |
| 256 | +<!-- markdownlint-restore --> |
| 257 | + |
| 258 | +```python |
| 259 | +from advanced_alchemy.extensions.fastapi import AdvancedAlchemy, SQLAlchemyAsyncConfig |
| 260 | +from fastapi import FastAPI |
| 261 | + |
| 262 | +app = FastAPI() |
| 263 | +alchemy = AdvancedAlchemy( |
| 264 | + config=SQLAlchemyAsyncConfig(connection_string="sqlite+aiosqlite:///test.sqlite"), app=app, |
| 265 | +) |
| 266 | +``` |
| 267 | + |
| 268 | +</details> |
| 269 | + |
| 270 | +For a full FastAPI example with optional CLI integration, see [here][fastapi-example] |
| 271 | + |
| 272 | +#### Starlette |
| 273 | + |
| 274 | +<!-- markdownlint-disable --> |
| 275 | +<details> |
| 276 | +<summary>Pre-built Example Apps</summary> |
| 277 | +<!-- markdownlint-restore --> |
| 278 | + |
| 279 | +```python |
| 280 | +from advanced_alchemy.extensions.starlette import AdvancedAlchemy, SQLAlchemyAsyncConfig |
| 281 | +from starlette.applications import Starlette |
| 282 | + |
| 283 | +app = Starlette() |
| 284 | +alchemy = AdvancedAlchemy( |
| 285 | + config=SQLAlchemyAsyncConfig(connection_string="sqlite+aiosqlite:///test.sqlite"), app=app, |
| 286 | +) |
| 287 | +``` |
| 288 | + |
| 289 | +</details> |
| 290 | + |
| 291 | +#### Sanic |
| 292 | + |
| 293 | +<!-- markdownlint-disable --> |
| 294 | +<details> |
| 295 | +<summary>Pre-built Example Apps</summary> |
| 296 | +<!-- markdownlint-restore --> |
| 297 | + |
| 298 | +```python |
| 299 | +from sanic import Sanic |
| 300 | +from sanic_ext import Extend |
| 301 | + |
| 302 | +from advanced_alchemy.extensions.sanic import AdvancedAlchemy, SQLAlchemyAsyncConfig |
| 303 | + |
| 304 | +app = Sanic("AlchemySanicApp") |
| 305 | +alchemy = AdvancedAlchemy( |
| 306 | + sqlalchemy_config=SQLAlchemyAsyncConfig(connection_string="sqlite+aiosqlite:///test.sqlite"), |
| 307 | +) |
| 308 | +Extend.register(alchemy) |
| 309 | +``` |
| 310 | + |
| 311 | +</details> |
| 312 | + |
| 313 | +## Contributing |
| 314 | + |
| 315 | +All [Litestar Organization][litestar-org] projects will always be a community-centered, available for contributions of any size. |
| 316 | + |
| 317 | +Before contributing, please review the [contribution guide][contributing]. |
| 318 | + |
| 319 | +If you have any questions, reach out to us on [Discord][discord], our org-wide [GitHub discussions][litestar-discussions] page, |
| 320 | +or the [project-specific GitHub discussions page][project-discussions]. |
| 321 | + |
| 322 | +<!-- markdownlint-disable --> |
| 323 | +<hr /> |
| 324 | +<p align="center"> |
| 325 | + |
| 326 | +</p> |
5 | 327 |
|
6 | 328 | [litestar-org]: https://github.com/litestar-org
|
7 | 329 | [contributing]: https://docs.advanced-alchemy.litestar.dev/latest/contribution-guide.html
|
|
0 commit comments