add tests for migrations.

Author: jayvynl

README.md

@@ -10,6 +10,7 @@ clickhouse.
 Thanks to [clickhouse driver](https://github.com/mymarilyn/clickhouse-driver), django clickhouse backend use it as [DBAPI](https://peps.python.org/pep-0249/).
 Thanks to [clickhouse pool](https://github.com/ericmccarthy7/clickhouse-pool), it makes clickhouse connection pool.
 
+[Documentation](https://github.com/jayvynl/django-clickhouse-backend/blob/main/docs/README.md)
 
 **features:**
 
@@ -162,62 +163,31 @@ Topics
 Django ORM depends heavily on single column primary key, this primary key is a unique identifier of an ORM object.
 All `get` `save` `delete` actions depend on primary key.
 
-But in ClickHouse [primary key](https://clickhouse.com/docs/en/engines/table-engines/mergetree-family/mergetree#primary-keys-and-indexes-in-queries) has different meaning with django primary key. ClickHouse does not require a unique primary key. You can insert multiple rows with the same primary key.
+But in ClickHouse [primary key](https://clickhouse.com/docs/en/engines/table-engines/mergetree-family/mergetree#primary-keys-and-indexes-in-queries) has different meaning from django primary key. ClickHouse does not require a unique primary key. You can insert multiple rows with the same primary key.
 
 There is [no unique constraint](https://github.com/ClickHouse/ClickHouse/issues/3386#issuecomment-429874647) or auto increasing column in clickhouse.
 
 By default, django will add a field named `id` as auto increasing primary key.
 
 - AutoField
 
-  Mapped to clickhouse Int32 data type. You should generate this unique id yourself
+  Mapped to clickhouse Int32 data type. You should generate this unique id yourself.
 
 - BigAutoField
 
-  Mapped to clickhouse Int64 data type. If primary key is not specified when insert data, then `clickhouse_driver.idworker.id_worker` is used to generate this unique key.
+  Mapped to clickhouse Int64 data type. If primary key is not specified when insert data, then `clickhouse_driver.idworker.id_worker` is used to generate this unique id.
 
   Default id_worker is an instance of `clickhouse.idworker.snowflake.SnowflakeIDWorker` which implement [twitter snowflake id](https://en.wikipedia.org/wiki/Snowflake_ID).
   If data insertions happen on multiple datacenter, server, process or thread, you should ensure uniqueness of (CLICKHOUSE_WORKER_ID, CLICKHOUSE_DATACENTER_ID) environment variable.
   Because work_id and datacenter_id are 5 bits, they should be an integer between 0 and 31. CLICKHOUSE_WORKER_ID default to 0, CLICKHOUSE_DATACENTER_ID will be generated randomly if not provided.
 
-  `clickhouse.idworker.snowflake.SnowflakeIDWorker` is not thread safe. You could inherit `clickhouse.idworker.base.BaseIDWorker` and implement one, and set `CLICKHOUSE_ID_WORKER` to doted import path of your IDWorker instance.
+  `clickhouse.idworker.snowflake.SnowflakeIDWorker` is not thread safe. You could inherit `clickhouse.idworker.base.BaseIDWorker` and implement one, then set `CLICKHOUSE_ID_WORKER` in `settings.py` to doted import path of your IDWorker instance.
 
 Django use a table named `django_migrations` to track migration files. ID field should be BigAutoField, so that IDWorker can generate unique id for you.
 After Django 3.2，a new [config `DEFAULT_AUTO_FIELD`](https://docs.djangoproject.com/en/4.1/releases/3.2/#customizing-type-of-auto-created-primary-keys) is introduced to control field type of default primary key.
 So `DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'` is required if you want to use migrations with django clickhouse backend.
 
 
-### Fields
-
-#### Nullable
-
-`null=True` will make [Nullable](https://clickhouse.com/docs/en/sql-reference/data-types/nullable/) type in clickhouse database.
-
-**Note** Using Nullable almost always negatively affects performance, keep this in mind when designing your databases.
-
-#### GenericIPAddressField
-
-Clickhouse backend has its own implementation in `clickhouse_backend.models.fields.GenericIPAddressField`.
-If `protocol='ipv4'`, a column of [IPv4](https://clickhouse.com/docs/en/sql-reference/data-types/domains/ipv4) is generated, else [IPv6](https://clickhouse.com/docs/en/sql-reference/data-types/domains/ipv6) is generated.
-
-#### PositiveSmallIntegerField
-#### PositiveIntegerField
-#### PositiveBigIntegerField
-
-`clickhouse_backend.models.fields.PositiveSmallIntegerField` maps to [UInt16](https://clickhouse.com/docs/en/sql-reference/data-types/int-uint).
-`clickhouse_backend.models.fields.PositiveIntegerField` maps to [UInt32](https://clickhouse.com/docs/en/sql-reference/data-types/int-uint).
-`clickhouse_backend.models.fields.PositiveBigIntegerField` maps to [UInt64](https://clickhouse.com/docs/en/sql-reference/data-types/int-uint).
-Clickhouse have unsigned integer type, these fields will have right integer range validators.
-
-
-### Engines
-
-Lays in `clickhouse_backend.models.engines`.
-
-### Indexes
-
-Lays in `clickouse_backend.models.indexes`.
-
 Test
 ---
 
@@ -231,7 +201,7 @@ docker-compose up -d
 python tests/runtests.py
 ```
 
-**Note** This project is not fully tested yet and should be used with caution in production.
+**Note:** This project is not fully tested yet and should be used with caution in production.
 
 License
 ---

clickhouse_backend/backend/base.py

@@ -25,6 +25,7 @@ class DatabaseWrapper(BaseDatabaseWrapper):
     # be interpolated against the values of Field.__dict__ before being output.
     # If a column type is set to None, it won't be included in the output.
     data_types = {
+        'SmallAutoField': 'Int16',
         'AutoField': 'Int32',
         'BigAutoField': 'Int64',
         'IPAddressField': 'IPv4',
@@ -37,13 +38,13 @@ class DatabaseWrapper(BaseDatabaseWrapper):
         'FileField': 'String',
         'FilePathField': 'String',
         'FloatField': 'Float64',
+        'SmallIntegerField': 'Int16',
         'IntegerField': 'Int32',
         'BigIntegerField': 'Int64',
         'PositiveBigIntegerField': 'UInt64',
         'PositiveIntegerField': 'UInt32',
         'PositiveSmallIntegerField': 'UInt16',
         'SlugField': 'String',
-        'SmallIntegerField': 'Int16',
         'TextField': 'String',
         'UUIDField': 'UUID',
         'BooleanField': 'Int8',
@@ -71,7 +72,7 @@ class DatabaseWrapper(BaseDatabaseWrapper):
     #
     # Note: we use str.format() here for readability as '%' is used as a wildcard for
     # the LIKE operator.
-    pattern_esc = r"replaceAll(replaceAll(replaceAll({}, '\\', '\\\\'), '%', '\\%'), '_', '\\_')"
+    pattern_esc = r"replaceRegexpAll({}, '\\\\|%%|_', '\\\\\\0')"
     pattern_ops = {
         'contains': "LIKE '%%' || {} || '%%'",
         'icontains': "ILIKE '%%' || {} || '%%'",

clickhouse_backend/backend/operations.py

@@ -4,6 +4,7 @@
 from django.db.backends.base.operations import BaseDatabaseOperations
 
 from clickhouse_backend import compat
+from clickhouse_backend.driver.client import insert_pattern
 
 
 class DatabaseOperations(BaseDatabaseOperations):
@@ -350,10 +351,11 @@ def savepoint_rollback_sql(self, sid):
         return super().savepoint_rollback_sql(sid)
 
     def last_executed_query(self, cursor, sql, params):
-        if sql.startswith("INSERT INTO"):
-            return "%s %s" % (sql, ", ".join(map(str, params)))
         if params:
-            return sql % params
+            if insert_pattern.match(sql):
+                return "%s %s" % (sql, str(tuple(tuple(param) for param in params)))
+            else:
+                return sql % tuple(params)
         return sql
 
     def settings_sql(self, **settings):