0.60.2

Fixed a bug with asyncio.gather not working with some query types. It was due to them being dataclasses, and they couldn't be hashed properly. Thanks to @brnosouza for reporting this issue.

0.60.1

Modified the import path for MigrationManager in migration files. It was confusing Pylance (VSCode's type checker). Thanks to @gmos for reporting and investigating this issue.

0.60.0

Secret columns

All column types can now be secret, rather than being limited to the Secret column type which is a Varchar under the hood (courtesy @sinisaos).

class Manager(Table):
    name = Varchar()
    net_worth = Integer(secret=True)

The reason this is useful is you can do queries such as:

>>> Manager.select(exclude_secrets=True).run_sync()
[{'id': 1, 'name': 'Guido'}]

In the Piccolo API project we have PiccoloCRUD which is an incredibly powerful way of building an API with very little code. PiccoloCRUD has an exclude_secrets option which lets you safely expose your data without leaking sensitive information.

Pydantic improvements

max_recursion_depth

create_pydantic_model now has a max_recursion_depth argument, which is useful when using nested=True on large database schemas.

>>> create_pydantic_model(MyTable, nested=True, max_recursion_depth=3)

Nested tuple

You can now pass a tuple of columns as the argument to nested:

>>> create_pydantic_model(Band, nested=(Band.manager,))

This gives you more control than just using nested=True.

include_columns / exclude_columns

You can now include / exclude columns from related tables. For example:

>>> create_pydantic_model(Band, nested=(Band.manager,), exclude_columns=(Band.manager.country))

Similarly:

>>> create_pydantic_model(Band, nested=(Band.manager,), include_columns=(Band.name, Band.manager.name))

0.59.0

• When using piccolo asgi new to generate a FastAPI app, the generated code is now cleaner. It also contains a conftest.py file, which encourages people to use piccolo tester run rather than using pytest directly.
• Tidied up docs, and added logo.
• Clarified the use of the PICCOLO_CONF environment variable in the docs (courtesy @theelderbeever).
• create_pydantic_model now accepts an include_columns argument, in case you only want a few columns in your model, it's faster than using exclude_columns (courtesy @sinisaos).
• Updated linters, and fixed new errors.

0.58.0

Improved Pydantic docs

The Pydantic docs used to be in the Piccolo API repo, but have been moved over to this repo. We took this opportunity to improve them significantly with additional examples. Courtesy @sinisaos.

Internal code refactoring

Some of the code has been optimised and cleaned up. Courtesy @yezz123.

Schema generation for recursive foreign keys

When using piccolo schema generate, it would get stuck in a loop if a table had a foreign key column which referenced itself. Thanks to @knguyen5 for reporting this issue, and @wmshort for implementing the fix. The output will now look like:

class Employee(Table):
    name = Varchar()
    manager = ForeignKey("self")

Fixing a bug with Alter.add_column

When using the Alter.add_column API directly (not via migrations), it would fail with foreign key columns. For example:

SomeTable.alter().add_column(
    name="my_fk_column",
    column=ForeignKey(SomeOtherTable)
).run_sync()

This has now been fixed. Thanks to @wmshort for discovering this issue.

create_pydantic_model improvements

Additional fields can now be added to the Pydantic schema. This is useful when using Pydantic's JSON schema functionality:

my_model = create_pydantic_model(Band, my_extra_field="Hello")
>>> my_model.schema()
{..., "my_extra_field": "Hello"}

This feature was added to support new features in Piccolo Admin.

Fixing a bug with import clashes in migrations

In certain situations it was possible to create a migration file with clashing imports. For example:

from uuid import UUID
from piccolo.columns.column_types import UUID

Piccolo now tries to detect these clashes, and prevent them. If they can't be prevented automatically, a warning is shown to the user. Courtesy @0scarB.

0.57.0

Added Python 3.10 support (courtesy @kennethcheo).

0.56.0

Fixed schema generation bug

When using piccolo schema generate to auto generate Piccolo Table classes from an existing database, it would fail in this situation:

• A table has a column with an index.
• The column name clashed with a Postgres type.

For example, we couldn't auto generate this Table class:

class MyTable(Table):
    time = Timestamp(index=True)

This is because time is a builtin Postgres type, and the CREATE INDEX statement being inspected in the database wrapped the column name in quotes, which broke our regex.

Thanks to @knguyen5 for fixing this.

Improved testing docs

A convenience method called get_table_classes was added to Finder.

Finder is the main class in Piccolo for dynamically importing projects / apps / tables / migrations etc.

get_table_classes lets us easily get the Table classes for a project. This makes writing unit tests easier, when we need to setup a schema.

from unittest import TestCase

from piccolo.table import create_tables, drop_tables
from piccolo.conf.apps import Finder

TABLES = Finder().get_table_classes()

class TestApp(TestCase):
    def setUp(self):
        create_tables(*TABLES)

    def tearDown(self):
        drop_tables(*TABLES)

    def test_app(self):
        # Do some testing ...
        pass

The docs were updated to reflect this.

When dropping tables in a unit test, remember to use piccolo tester run, to make sure the test database is used.

get_output_schema

get_output_schema is the main entrypoint for database reflection in Piccolo. It has been modified to accept an optional Engine argument, which makes it more flexible.

0.55.0

Table._meta.refresh_db

Added the ability to refresh the database engine.

MyTable._meta.refresh_db()

This causes the Table to fetch the Engine again from your piccolo_conf.py file. The reason this is useful, is you might change the PICCOLO_CONF environment variable, and some Table classes have already imported an engine. This is now used by the piccolo tester run command to ensure all Table classes have the correct engine.

ColumnMeta edge cases

Fixed an edge case where ColumnMeta couldn't be copied if it had extra attributes added to it.

Improved column type conversion

When running migrations which change column types, Piccolo now provides the USING clause to the ALTER COLUMN DDL statement, which makes it more likely that type conversion will be successful.

For example, if there is an Integer column, and it's converted to a Varchar column, the migration will run fine. In the past, running this in reverse would fail. Now Postgres will try and cast the values back to integers, which makes reversing migrations more likely to succeed.

Added drop_tables

There is now a convenience function for dropping several tables in one go. If the database doesn't support CASCADE, then the tables are sorted based on their ForeignKey columns, so they're dropped in the correct order. It all runs inside a transaction.

from piccolo.table import drop_tables

drop_tables(Band, Manager)

This is a useful tool in unit tests.

Index support in schema generation

When using piccolo schema generate, Piccolo will now reflect the indexes from the database into the generated Table classes. Thanks to @wmshort for this.

0.54.0

Added the db_column_name option to columns. This is for edge cases where a legacy database is being used, with problematic column names. For example, if a column is called class, this clashes with a Python builtin, so the following isn't possible:

class MyTable(Table):
    class = Varchar()  # Syntax error!

You can now do the following:

class MyTable(Table):
    class_ = Varchar(db_column_name='class')

Here are some example queries using it:

# Create - both work as expected
MyTable(class_='Test').save().run_sync()
MyTable.objects().create(class_='Test').run_sync()

# Objects
row = MyTable.objects().first().where(MyTable.class_ == 'Test').run_sync()
>>> row.class_
'Test'

# Select
>>> MyTable.select().first().where(MyTable.class_ == 'Test').run_sync()
{'id': 1, 'class': 'Test'}

0.53.0

An internal code clean up (courtesy @yezz123).

Dramatically improved CLI appearance when running migrations (courtesy @wmshort).

Added a runtime reflection feature, where Table classes can be generated on the fly from existing database tables (courtesy @AliSayyah). This is useful when dealing with very dynamic databases, where tables are frequently being added / modified, so hard coding them in a tables.py file is impractical. Also, for exploring databases on the command line. It currently just supports Postgres.

Here's an example:

from piccolo.table_reflection import TableStorage

storage = TableStorage()
Band = await storage.get_table('band')
>>> await Band.select().run()
[{'id': 1, 'name': 'Pythonistas', 'manager': 1}, ...]