Skip to content

Commit 8271c86

Browse files
Clean up dead code and outdated terminology
Code cleanup: - Remove backward compatibility aliases (ObjectType, AttachType, etc.) - Remove misleading comments about non-existent DJBlobType/ContentType - Remove unused build_foreign_key_parser_old function - Remove unused feature switches (ADAPTED_TYPE_SWITCH, FILEPATH_FEATURE_SWITCH) - Remove unused os import from errors.py - Rename ADAPTED type category to CODEC Documentation fixes: - Update mkdocs.yaml nav: customtype.md → codecs.md - Fix dead links in attributes.md pointing to customtype.md Terminology updates: - Replace "AttributeType" with "Codec" in all comments - Replace "Adapter" with "Codec" in docstrings - Fix SHA256 → MD5 in content_registry.py docstring Version bump to 2.0.0a6 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
1 parent 568d71a commit 8271c86

File tree

11 files changed

+19
-102
lines changed

11 files changed

+19
-102
lines changed

docs/mkdocs.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ nav:
3333
- Blobs: design/tables/blobs.md
3434
- Attachments: design/tables/attach.md
3535
- Filepaths: design/tables/filepath.md
36-
- Custom Datatypes: design/tables/customtype.md
36+
- Custom Codecs: design/tables/codecs.md
3737
- Dependencies: design/tables/dependencies.md
3838
- Indexes: design/tables/indexes.md
3939
- Master-Part Relationships: design/tables/master-part.md

docs/src/design/tables/attributes.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -96,7 +96,7 @@ The `@` character indicates **external storage** (object store vs database):
9696

9797
- `<blob>`: DataJoint's native serialization format for Python objects. Supports
9898
NumPy arrays, dicts, lists, datetime objects, and nested structures. Stores in
99-
database. Compatible with MATLAB. See [custom types](customtype.md) for details.
99+
database. Compatible with MATLAB. See [custom codecs](codecs.md) for details.
100100

101101
- `<blob@>` / `<blob@store>`: Like `<blob>` but stores externally with hash-
102102
addressed deduplication. Use for large arrays that may be duplicated across rows.
@@ -125,7 +125,7 @@ The `@` character indicates **external storage** (object store vs database):
125125

126126
### User-defined codecs
127127

128-
- `<custom_type>`: Define your own [custom codec](customtype.md) with
128+
- `<custom_type>`: Define your own [custom codec](codecs.md) with
129129
bidirectional conversion between Python objects and database storage. Use for
130130
graphs, domain-specific objects, or custom data structures.
131131

src/datajoint/builtin_codecs.py

Lines changed: 0 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -119,9 +119,6 @@ def decode(self, stored: bytes, *, key: dict | None = None) -> Any:
119119
return blob.unpack(stored, squeeze=False)
120120

121121

122-
# Note: DJBlobType is defined at end of file as DJBlobCodec (not BlobCodec)
123-
124-
125122
# =============================================================================
126123
# Hash-Addressed Storage Codec
127124
# =============================================================================
@@ -203,9 +200,6 @@ def validate(self, value: Any) -> None:
203200
raise TypeError(f"<hash> expects bytes, got {type(value).__name__}")
204201

205202

206-
# Note: ContentType is defined at end of file as ContentCodec (not HashCodec)
207-
208-
209203
# =============================================================================
210204
# Path-Addressed Storage Codec (OAS - Object-Augmented Schema)
211205
# =============================================================================
@@ -419,10 +413,6 @@ def validate(self, value: Any) -> None:
419413
raise TypeError(f"<object> expects bytes or path, got {type(value).__name__}")
420414

421415

422-
# Backward compatibility alias
423-
ObjectType = ObjectCodec
424-
425-
426416
# =============================================================================
427417
# File Attachment Codecs
428418
# =============================================================================
@@ -544,11 +534,6 @@ def validate(self, value: Any) -> None:
544534
raise TypeError(f"<attach> expects a file path, got {type(value).__name__}")
545535

546536

547-
# Backward compatibility aliases
548-
AttachType = AttachCodec
549-
XAttachType = AttachCodec # <attach@> is now just AttachCodec with external storage
550-
551-
552537
# =============================================================================
553538
# Filepath Reference Codec
554539
# =============================================================================
@@ -664,7 +649,3 @@ def validate(self, value: Any) -> None:
664649

665650
if not isinstance(value, (str, Path)):
666651
raise TypeError(f"<filepath> expects a path string or Path, got {type(value).__name__}")
667-
668-
669-
# Backward compatibility alias
670-
FilepathType = FilepathCodec

src/datajoint/content_registry.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,8 @@
22
Content-addressed storage registry for DataJoint.
33
44
This module provides content-addressed storage with deduplication for the <hash>
5-
AttributeType. Content is identified by its SHA256 hash and stored in a hierarchical
6-
directory structure: _content/{hash[:2]}/{hash[2:4]}/{hash}
5+
Codec. Content is identified by its MD5 hash and stored in a hierarchical
6+
directory structure: _hash/{hash[:2]}/{hash[2:4]}/{hash}
77
88
The ContentRegistry tracks stored content for garbage collection purposes.
99
"""

src/datajoint/declare.py

Lines changed: 6 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -76,16 +76,16 @@
7676
TEMPORAL=r"(time|timestamp|year)(\s*\(.+\))?$", # time, timestamp, year (not date/datetime)
7777
NATIVE_BLOB=r"(tiny|small|medium|long)blob$", # Specific blob variants
7878
NATIVE_TEXT=r"(tiny|small|medium|long)text$", # Text variants (use plain 'text' instead)
79-
# AttributeTypes use angle brackets
80-
ADAPTED=r"<.+>$",
79+
# Codecs use angle brackets
80+
CODEC=r"<.+>$",
8181
).items()
8282
}
8383

8484
# Core types are stored in attribute comment for reconstruction
8585
CORE_TYPE_NAMES = {name.upper() for name in CORE_TYPES}
8686

8787
# Special types that need comment storage (core types + adapted)
88-
SPECIAL_TYPES = CORE_TYPE_NAMES | {"ADAPTED"}
88+
SPECIAL_TYPES = CORE_TYPE_NAMES | {"CODEC"}
8989

9090
# Native SQL types that pass through (with optional warning)
9191
NATIVE_TYPES = set(TYPE_PATTERN) - SPECIAL_TYPES
@@ -104,23 +104,6 @@ def match_type(attribute_type):
104104
logger = logging.getLogger(__name__.split(".")[0])
105105

106106

107-
def build_foreign_key_parser_old():
108-
# old-style foreign key parser. Superseded by expression-based syntax. See issue #436
109-
# This will be deprecated in a future release.
110-
left = pp.Literal("(").suppress()
111-
right = pp.Literal(")").suppress()
112-
attribute_name = pp.Word(pp.srange("[a-z]"), pp.srange("[a-z0-9_]"))
113-
new_attrs = pp.Optional(left + pp.DelimitedList(attribute_name) + right).set_results_name("new_attrs")
114-
arrow = pp.Literal("->").suppress()
115-
lbracket = pp.Literal("[").suppress()
116-
rbracket = pp.Literal("]").suppress()
117-
option = pp.Word(pp.srange("[a-zA-Z]"))
118-
options = pp.Optional(lbracket + pp.DelimitedList(option) + rbracket).set_results_name("options")
119-
ref_table = pp.Word(pp.alphas, pp.alphanums + "._").set_results_name("ref_table")
120-
ref_attrs = pp.Optional(left + pp.DelimitedList(attribute_name) + right).set_results_name("ref_attrs")
121-
return new_attrs + arrow + options + ref_table + ref_attrs
122-
123-
124107
def build_foreign_key_parser():
125108
arrow = pp.Literal("->").suppress()
126109
lbracket = pp.Literal("[").suppress()
@@ -144,7 +127,6 @@ def build_attribute_parser():
144127
return attribute_name + pp.Optional(default) + colon + data_type + comment
145128

146129

147-
foreign_key_parser_old = build_foreign_key_parser_old()
148130
foreign_key_parser = build_foreign_key_parser()
149131
attribute_parser = build_attribute_parser()
150132

@@ -459,14 +441,14 @@ def substitute_special_type(match, category, foreign_key_sql, context):
459441
460442
Special types are:
461443
- Core DataJoint types (float32 → float, uuid → binary(16), bytes → longblob, etc.)
462-
- ADAPTED types (AttributeTypes in angle brackets)
444+
- CODEC types (Codecs in angle brackets)
463445
464446
:param match: dict containing with keys "type" and "comment" -- will be modified in place
465447
:param category: attribute type category from TYPE_PATTERN
466448
:param foreign_key_sql: list of foreign key declarations to add to
467449
:param context: context for looking up user-defined codecs (unused, kept for compatibility)
468450
"""
469-
if category == "ADAPTED":
451+
if category == "CODEC":
470452
# Codec - resolve to underlying dtype
471453
codec, store_name = lookup_codec(match["type"])
472454
if store_name is not None:
@@ -540,7 +522,7 @@ def compile_attribute(line, in_key, foreign_key_sql, context):
540522
category = match_type(match["type"])
541523

542524
if category in SPECIAL_TYPES:
543-
# Core types and AttributeTypes are recorded in comment for reconstruction
525+
# Core types and Codecs are recorded in comment for reconstruction
544526
match["comment"] = ":{type}:{comment}".format(**match)
545527
substitute_special_type(match, category, foreign_key_sql, context)
546528
elif category in NATIVE_TYPES:

src/datajoint/errors.py

Lines changed: 0 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,6 @@
22
Exception classes for the DataJoint library
33
"""
44

5-
import os
6-
75

86
# --- Top Level ---
97
class DataJointError(Exception):
@@ -87,43 +85,3 @@ class BucketInaccessible(DataJointError):
8785
"""
8886
Error raised when a S3 bucket is inaccessible
8987
"""
90-
91-
92-
# environment variables to control availability of experimental features
93-
94-
ADAPTED_TYPE_SWITCH = "DJ_SUPPORT_ADAPTED_TYPES"
95-
FILEPATH_FEATURE_SWITCH = "DJ_SUPPORT_FILEPATH_MANAGEMENT"
96-
97-
98-
def _switch_adapted_types(on):
99-
"""
100-
Enable (on=True) or disable (on=False) support for AttributeAdapter
101-
"""
102-
if on:
103-
os.environ[ADAPTED_TYPE_SWITCH] = "TRUE"
104-
else:
105-
del os.environ[ADAPTED_TYPE_SWITCH]
106-
107-
108-
def _support_adapted_types():
109-
"""
110-
check if support for AttributeAdapter is enabled
111-
"""
112-
return os.getenv(ADAPTED_TYPE_SWITCH, "FALSE").upper() == "TRUE"
113-
114-
115-
def _switch_filepath_types(on):
116-
"""
117-
Enable (on=True) or disable (on=False) support for AttributeAdapter
118-
"""
119-
if on:
120-
os.environ[FILEPATH_FEATURE_SWITCH] = "TRUE"
121-
else:
122-
del os.environ[FILEPATH_FEATURE_SWITCH]
123-
124-
125-
def _support_filepath_types():
126-
"""
127-
check if support for AttributeAdapter is enabled
128-
"""
129-
return os.getenv(FILEPATH_FEATURE_SWITCH, "FALSE").upper() == "TRUE"

src/datajoint/heading.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -307,8 +307,8 @@ def _init_from_database(self):
307307
# Store the original type name for display but keep db_type for SQL
308308
attr["original_type"] = special["type"]
309309

310-
# process Codecs (adapted types in angle brackets)
311-
if special and TYPE_PATTERN["ADAPTED"].match(attr["type"]):
310+
# process Codecs (types in angle brackets)
311+
if special and TYPE_PATTERN["CODEC"].match(attr["type"]):
312312
# Context can be None for built-in types that are globally registered
313313
codec_spec = special["type"]
314314
try:

src/datajoint/migrate.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
Migration utilities for DataJoint schema updates.
33
44
This module provides tools for migrating existing schemas to use the new
5-
AttributeType system, particularly for upgrading blob columns to use
5+
Codec system, particularly for upgrading blob columns to use
66
explicit `<blob>` type declarations.
77
"""
88

src/datajoint/table.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -732,7 +732,7 @@ def __make_placeholder(self, name, value, ignore_extra_fields=False, row=None):
732732
processing by mysql API.
733733
734734
In the simplified type system:
735-
- Adapters (AttributeTypes) handle all custom encoding via type chains
735+
- Codecs handle all custom encoding via type chains
736736
- UUID values are converted to bytes
737737
- JSON values are serialized
738738
- Blob values pass through as bytes

src/datajoint/version.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
# version bump auto managed by Github Actions:
22
# label_prs.yaml(prep), release.yaml(bump), post_release.yaml(edit)
33
# manually set this version will be eventually overwritten by the above actions
4-
__version__ = "2.0.0a5"
4+
__version__ = "2.0.0a6"

0 commit comments

Comments
 (0)