✨ Add support for Decimal fields from Pydantic and SQLAlchemy (#103)

Co-authored-by: Sebastián Ramírez <[email protected]>

Author: robcxyz

docs/advanced/decimal.md

@@ -0,0 +1,148 @@
+# Decimal Numbers
+
+In some cases you might need to be able to store decimal numbers with guarantees about the precision.
+
+This is particularly important if you are storing things like **currencies**, **prices**, **accounts**, and others, as you would want to know that you wouldn't have rounding errors.
+
+As an example, if you open Python and sum `1.1` + `2.2` you would expect to see `3.3`, but you will actually get `3.3000000000000003`:
+
+```Python
+>>> 1.1 + 2.2
+3.3000000000000003
+```
+
+This is because of the way numbers are stored in "ones and zeros" (binary). But Python has a module and some types to have strict decimal values. You can read more about it in the official <a href="https://docs.python.org/3/library/decimal.html" class="external-link" target="_blank">Python docs for Decimal</a>.
+
+Because databases store data in the same ways as computers (in binary), they would have the same types of issues. And because of that, they also have a special **decimal** type.
+
+In most cases this would probably not be a problem, for example measuring views in a video, or the life bar in a videogame. But as you can imagine, this is particularly important when dealing with **money** and **finances**.
+
+## Decimal Types
+
+Pydantic has special support for `Decimal` types using the <a href="https://pydantic-docs.helpmanual.io/usage/types/#arguments-to-condecimal" class="external-link" target="_blank">`condecimal()` special function</a>.
+
+!!! tip
+    Pydantic 1.9, that will be released soon, has improved support for `Decimal` types, without needing to use the `condecimal()` function.
+
+    But meanwhile, you can already use this feature with `condecimal()` in **SQLModel** it as it's explained here.
+
+When you use `condecimal()` you can specify the number of digits and decimal places to support. They will be validated by Pydantic (for example when using FastAPI) and the same information will also be used for the database columns.
+
+!!! info
+    For the database, **SQLModel** will use <a href="https://docs.sqlalchemy.org/en/14/core/type_basics.html#sqlalchemy.types.DECIMAL" class="external-link" target="_blank">SQLAlchemy's `DECIMAL` type</a>.
+
+## Decimals in SQLModel
+
+Let's say that each hero in the database will have an amount of money. We could make that field a `Decimal` type using the `condecimal()` function:
+
+```{.python .annotate hl_lines="12" }
+{!./docs_src/advanced/decimal/tutorial001.py[ln:1-12]!}
+
+# More code here later 👇
+```
+
+<details>
+<summary>👀 Full file preview</summary>
+
+```Python
+{!./docs_src/advanced/decimal/tutorial001.py!}
+```
+
+</details>
+
+Here we are saying that `money` can have at most `5` digits with `max_digits`, **this includes the integers** (to the left of the decimal dot) **and the decimals** (to the right of the decimal dot).
+
+We are also saying that the number of decimal places (to the right of the decimal dot) is `3`, so we can have **3 decimal digits** for these numbers in the `money` field. This means that we will have **2 digits for the integer part** and **3 digits for the decimal part**.
+
+✅ So, for example, these are all valid numbers for the `money` field:
+
+* `12.345`
+* `12.3`
+* `12`
+* `1.2`
+* `0.123`
+* `0`
+
+🚫 But these are all invalid numbers for that `money` field:
+
+* `1.2345`
+    * This number has more than 3 decimal places.
+* `123.234`
+    * This number has more than 5 digits in total (integer and decimal part).
+* `123`
+    * Even though this number doesn't have any decimals, we still have 3 places saved for them, which means that we can **only use 2 places** for the **integer part**, and this number has 3 integer digits. So, the allowed number of integer digits is `max_digits` - `decimal_places` = 2.
+
+!!! tip
+    Make sure you adjust the number of digits and decimal places for your own needs, in your own application. 🤓
+
+## Create models with Decimals
+
+When creating new models you can actually pass normal (`float`) numbers, Pydantic will automatically convert them to `Decimal` types, and **SQLModel** will store them as `Decimal` types in the database (using SQLAlchemy).
+
+```Python hl_lines="4-6"
+# Code above omitted 👆
+
+{!./docs_src/advanced/decimal/tutorial001.py[ln:25-35]!}
+
+# Code below omitted 👇
+```
+
+<details>
+<summary>👀 Full file preview</summary>
+
+```Python
+{!./docs_src/advanced/decimal/tutorial001.py!}
+```
+
+</details>
+
+## Select Decimal data
+
+Then, when working with Decimal types, you can confirm that they indeed avoid those rounding errors from floats:
+
+```Python hl_lines="15-16"
+# Code above omitted 👆
+
+{!./docs_src/advanced/decimal/tutorial001.py[ln:38-51]!}
+
+# Code below omitted 👇
+```
+
+<details>
+<summary>👀 Full file preview</summary>
+
+```Python
+{!./docs_src/advanced/decimal/tutorial001.py!}
+```
+
+</details>
+
+## Review the results
+
+Now if you run this, instead of printing the unexpected number `3.3000000000000003`, it prints `3.300`:
+
+<div class="termy">
+
+```console
+$ python app.py
+
+// Some boilerplate and previous output omitted 😉
+
+// The type of money is Decimal('1.100')
+Hero 1: id=1 secret_name='Dive Wilson' age=None name='Deadpond' money=Decimal('1.100')
+
+// More output omitted here 🤓
+
+// The type of money is Decimal('1.100')
+Hero 2: id=3 secret_name='Tommy Sharp' age=48 name='Rusty-Man' money=Decimal('2.200')
+
+// No rounding errors, just 3.3! 🎉
+Total money: 3.300
+```
+
+</div>
+
+!!! warning
+    Although Decimal types are supported and used in the Python side, not all databases support it. In particular, SQLite doesn't support decimals, so it will convert them to the same floating `NUMERIC` type it supports.
+
+    But decimals are supported by most of the other SQL databases. 🎉

docs/advanced/index.md

@@ -1,12 +1,10 @@
 # Advanced User Guide
 
-The **Advanced User Guide** will be coming soon to a <del>theater</del> **documentation** near you... 😅
+The **Advanced User Guide** is gradually growing, you can already read about some advanced topics.
 
-I just have to `add` it, `commit` it, and `refresh` it. 😉
+At some point it will include:
 
-It will include:
-
-* How to use the `async` and `await` with the async session.
+* How to use `async` and `await` with the async session.
 * How to run migrations.
 * How to combine **SQLModel** models with SQLAlchemy.
-* ...and more.
+* ...and more. 🤓

docs_src/advanced/__init__.py

@@ -0,0 +1,61 @@
+from typing import Optional
+
+from pydantic import condecimal
+from sqlmodel import Field, Session, SQLModel, create_engine, select
+
+
+class Hero(SQLModel, table=True):
+    id: Optional[int] = Field(default=None, primary_key=True)
+    name: str
+    secret_name: str
+    age: Optional[int] = None
+    money: condecimal(max_digits=6, decimal_places=3) = Field(default=0)
+
+
+sqlite_file_name = "database.db"
+sqlite_url = f"sqlite:///{sqlite_file_name}"
+
+engine = create_engine(sqlite_url, echo=True)
+
+
+def create_db_and_tables():
+    SQLModel.metadata.create_all(engine)
+
+
+def create_heroes():
+    hero_1 = Hero(name="Deadpond", secret_name="Dive Wilson", money=1.1)
+    hero_2 = Hero(name="Spider-Boy", secret_name="Pedro Parqueador", money=0.001)
+    hero_3 = Hero(name="Rusty-Man", secret_name="Tommy Sharp", age=48, money=2.2)
+
+    with Session(engine) as session:
+        session.add(hero_1)
+        session.add(hero_2)
+        session.add(hero_3)
+
+        session.commit()
+
+
+def select_heroes():
+    with Session(engine) as session:
+        statement = select(Hero).where(Hero.name == "Deadpond")
+        results = session.exec(statement)
+        hero_1 = results.one()
+        print("Hero 1:", hero_1)
+
+        statement = select(Hero).where(Hero.name == "Rusty-Man")
+        results = session.exec(statement)
+        hero_2 = results.one()
+        print("Hero 2:", hero_2)
+
+        total_money = hero_1.money + hero_2.money
+        print(f"Total money: {total_money}")
+
+
+def main():
+    create_db_and_tables()
+    create_heroes()
+    select_heroes()
+
+
+if __name__ == "__main__":
+    main()

docs_src/advanced/decimal/__init__.py

@@ -84,6 +84,7 @@ nav:
       - tutorial/fastapi/tests.md
   - Advanced User Guide:
     - advanced/index.md
+    - advanced/decimal.md
   - alternatives.md
   - help.md
   - contributing.md

docs_src/advanced/decimal/tutorial001.py

@@ -399,7 +399,10 @@ def get_sqlachemy_type(field: ModelField) -> Any:
     if issubclass(field.type_, bytes):
         return LargeBinary
     if issubclass(field.type_, Decimal):
-        return Numeric
+        return Numeric(
+            precision=getattr(field.type_, "max_digits", None),
+            scale=getattr(field.type_, "decimal_places", None),
+        )
     if issubclass(field.type_, ipaddress.IPv4Address):
         return AutoString
     if issubclass(field.type_, ipaddress.IPv4Network):

mkdocs.yml

@@ -0,0 +1,44 @@
+from decimal import Decimal
+from unittest.mock import patch
+
+from sqlmodel import create_engine
+
+from ...conftest import get_testing_print_function
+
+expected_calls = [
+    [
+        "Hero 1:",
+        {
+            "name": "Deadpond",
+            "age": None,
+            "id": 1,
+            "secret_name": "Dive Wilson",
+            "money": Decimal("1.100"),
+        },
+    ],
+    [
+        "Hero 2:",
+        {
+            "name": "Rusty-Man",
+            "age": 48,
+            "id": 3,
+            "secret_name": "Tommy Sharp",
+            "money": Decimal("2.200"),
+        },
+    ],
+    ["Total money: 3.300"],
+]
+
+
+def test_tutorial(clear_sqlmodel):
+    from docs_src.advanced.decimal import tutorial001 as mod
+
+    mod.sqlite_url = "sqlite://"
+    mod.engine = create_engine(mod.sqlite_url)
+    calls = []
+
+    new_print = get_testing_print_function(calls)
+
+    with patch("builtins.print", new=new_print):
+        mod.main()
+    assert calls == expected_calls