Add DJBlobType and migration utilities for blob columns

Introduces `<djblob>` as an explicit AttributeType for DataJoint's
native blob serialization, allowing users to be explicit about
serialization behavior in table definitions.

Key changes:
- Add DJBlobType class with `serializes=True` flag to indicate
  it handles its own serialization (avoiding double pack/unpack)
- Update table.py and fetch.py to respect the `serializes` flag,
  skipping blob.pack/unpack when adapter handles serialization
- Add `dj.migrate` module with utilities for migrating existing
  schemas to use explicit `<djblob>` type declarations
- Add tests for DJBlobType functionality
- Document `<djblob>` type and migration procedure

The migration is metadata-only - blob data format is unchanged.
Existing `longblob` columns continue to work with implicit
serialization for backward compatibility.

Author: claude

docs/src/design/tables/customtype.md

@@ -476,3 +476,117 @@ def test_graph_type_roundtrip():
 
     assert set(g.edges) == set(decoded.edges)
 ```
+
+## Built-in Types
+
+DataJoint includes a built-in type for explicit blob serialization:
+
+### `<djblob>` - DataJoint Blob Serialization
+
+The `<djblob>` type provides explicit control over DataJoint's native binary
+serialization. It supports:
+
+- NumPy arrays (compatible with MATLAB)
+- Python dicts, lists, tuples, sets
+- datetime objects, Decimals, UUIDs
+- Nested data structures
+- Optional compression
+
+```python
+@schema
+class ProcessedData(dj.Manual):
+    definition = """
+    data_id : int
+    ---
+    results : <djblob>      # Explicit serialization
+    raw_bytes : longblob    # Backward-compatible (auto-serialized)
+    """
+```
+
+#### When to Use `<djblob>`
+
+- **New tables**: Prefer `<djblob>` for clarity and future-proofing
+- **Custom types**: Use `<djblob>` when your type chains to blob storage
+- **Migration**: Existing `longblob` columns can be migrated to `<djblob>`
+
+#### Backward Compatibility
+
+For backward compatibility, `longblob` columns without an explicit type
+still receive automatic serialization. The behavior is identical to `<djblob>`,
+but using `<djblob>` makes the serialization explicit in your code.
+
+## Schema Migration
+
+When upgrading existing schemas to use explicit type declarations, DataJoint
+provides migration utilities.
+
+### Analyzing Blob Columns
+
+```python
+import datajoint as dj
+
+schema = dj.schema("my_database")
+
+# Check migration status
+status = dj.migrate.check_migration_status(schema)
+print(f"Blob columns: {status['total_blob_columns']}")
+print(f"Already migrated: {status['migrated']}")
+print(f"Pending migration: {status['pending']}")
+```
+
+### Generating Migration SQL
+
+```python
+# Preview migration (dry run)
+result = dj.migrate.migrate_blob_columns(schema, dry_run=True)
+for sql in result['sql_statements']:
+    print(sql)
+```
+
+### Applying Migration
+
+```python
+# Apply migration
+result = dj.migrate.migrate_blob_columns(schema, dry_run=False)
+print(f"Migrated {result['migrated']} columns")
+```
+
+### Migration Details
+
+The migration updates MySQL column comments to include the type declaration.
+This is a **metadata-only** change - the actual blob data format is unchanged.
+
+Before migration:
+- Column: `longblob`
+- Comment: `user comment`
+- Behavior: Auto-serialization (implicit)
+
+After migration:
+- Column: `longblob`
+- Comment: `:<djblob>:user comment`
+- Behavior: Explicit serialization via `<djblob>`
+
+### Updating Table Definitions
+
+After database migration, update your Python table definitions for consistency:
+
+```python
+# Before
+class MyTable(dj.Manual):
+    definition = """
+    id : int
+    ---
+    data : longblob  # stored data
+    """
+
+# After
+class MyTable(dj.Manual):
+    definition = """
+    id : int
+    ---
+    data : <djblob>  # stored data
+    """
+```
+
+Both definitions work identically after migration, but using `<djblob>` makes
+the serialization explicit and documents the intended behavior.

src/datajoint/__init__.py

@@ -58,6 +58,7 @@
 ]
 
 from . import errors
+from . import migrate
 from .admin import kill
 from .attribute_adapter import AttributeAdapter
 from .attribute_type import AttributeType, list_types, register_type

src/datajoint/attribute_type.py

@@ -153,6 +153,10 @@ def decode(self, stored: Any, *, key: dict | None = None) -> Any:
         """
         ...
 
+    # Class attribute: If True, encode() produces final binary data (no blob.pack needed)
+    # Override in subclasses that handle their own serialization
+    serializes: bool = False
+
     def validate(self, value: Any) -> None:
         """
         Validate a value before encoding.
@@ -409,3 +413,124 @@ def resolve_dtype(dtype: str, seen: set[str] | None = None) -> tuple[str, list[A
 
     # Not a custom type - return as-is
     return dtype, chain
+
+
+# =============================================================================
+# Built-in Attribute Types
+# =============================================================================
+
+
+class DJBlobType(AttributeType):
+    """
+    Built-in type for DataJoint's native serialization format.
+
+    This type handles serialization of arbitrary Python objects (including NumPy arrays,
+    dictionaries, lists, etc.) using DataJoint's binary blob format. The format includes:
+
+    - Protocol headers (``mYm`` for MATLAB-compatible, ``dj0`` for Python-native)
+    - Optional compression (zlib)
+    - Support for NumPy arrays, datetime objects, UUIDs, and nested structures
+
+    The ``<djblob>`` type is the explicit way to specify DataJoint's serialization.
+    It stores data in a MySQL ``LONGBLOB`` column.
+
+    Example:
+        @schema
+        class ProcessedData(dj.Manual):
+            definition = '''
+            data_id : int
+            ---
+            results : <djblob>      # Explicit DataJoint serialization
+            raw_bytes : longblob    # Raw bytes (no serialization)
+            '''
+
+    Note:
+        For backward compatibility, ``longblob`` columns without an explicit type
+        still use automatic serialization. Use ``<djblob>`` to be explicit about
+        serialization behavior.
+    """
+
+    type_name = "djblob"
+    dtype = "longblob"
+    serializes = True  # This type handles its own serialization
+
+    def encode(self, value: Any, *, key: dict | None = None) -> bytes:
+        """
+        Serialize a Python object to DataJoint's blob format.
+
+        Args:
+            value: Any serializable Python object (dict, list, numpy array, etc.)
+            key: Primary key values (unused for blob serialization).
+
+        Returns:
+            Serialized bytes with protocol header and optional compression.
+        """
+        from . import blob
+
+        return blob.pack(value, compress=True)
+
+    def decode(self, stored: bytes, *, key: dict | None = None) -> Any:
+        """
+        Deserialize DataJoint blob format back to a Python object.
+
+        Args:
+            stored: Serialized blob bytes.
+            key: Primary key values (unused for blob serialization).
+
+        Returns:
+            The deserialized Python object.
+        """
+        from . import blob
+
+        return blob.unpack(stored, squeeze=False)
+
+
+class DJBlobExternalType(AttributeType):
+    """
+    Built-in type for externally-stored DataJoint blobs.
+
+    Similar to ``<djblob>`` but stores data in external blob storage instead
+    of inline in the database. Useful for large objects.
+
+    The store name is specified when defining the column type.
+
+    Example:
+        @schema
+        class LargeData(dj.Manual):
+            definition = '''
+            data_id : int
+            ---
+            large_array : blob@mystore  # External storage with auto-serialization
+            '''
+    """
+
+    # Note: This type isn't directly usable via <djblob_external> syntax
+    # It's used internally when blob@store syntax is detected
+    type_name = "djblob_external"
+    dtype = "blob@store"  # Placeholder - actual store is determined at declaration time
+    serializes = True  # This type handles its own serialization
+
+    def encode(self, value: Any, *, key: dict | None = None) -> bytes:
+        """Serialize a Python object to DataJoint's blob format."""
+        from . import blob
+
+        return blob.pack(value, compress=True)
+
+    def decode(self, stored: bytes, *, key: dict | None = None) -> Any:
+        """Deserialize DataJoint blob format back to a Python object."""
+        from . import blob
+
+        return blob.unpack(stored, squeeze=False)
+
+
+def _register_builtin_types() -> None:
+    """
+    Register DataJoint's built-in attribute types.
+
+    Called automatically during module initialization.
+    """
+    register_type(DJBlobType)
+
+
+# Register built-in types when module is loaded
+_register_builtin_types()

src/datajoint/fetch.py

@@ -88,18 +88,16 @@ def adapt(x):
             safe_write(local_filepath, data.split(b"\0", 1)[1])
         return adapt(str(local_filepath))  # download file from remote store
 
-    return adapt(
-        uuid.UUID(bytes=data)
-        if attr.uuid
-        else (
-            blob.unpack(
-                extern.get(uuid.UUID(bytes=data)) if attr.is_external else data,
-                squeeze=squeeze,
-            )
-            if attr.is_blob
-            else data
-        )
-    )
+    if attr.uuid:
+        return adapt(uuid.UUID(bytes=data))
+    elif attr.is_blob:
+        blob_data = extern.get(uuid.UUID(bytes=data)) if attr.is_external else data
+        # Skip unpack if adapter handles its own deserialization
+        if attr.adapter and getattr(attr.adapter, "serializes", False):
+            return attr.adapter.decode(blob_data, key=None)
+        return adapt(blob.unpack(blob_data, squeeze=squeeze))
+    else:
+        return adapt(data)
 
 
 class Fetch: