Skip to content

Commit 8d7c92e

Browse files
feat: Add NpyCodec for lazy-loading numpy arrays
Implement the `<npy@>` codec for schema-addressed numpy array storage: - Add SchemaCodec base class for path-addressed storage codecs - Add NpyRef class for lazy array references with metadata - Add NpyCodec using .npy format with shape/dtype inspection - Refactor ObjectCodec to inherit from SchemaCodec - Rename is_external to is_store throughout codebase - Export SchemaCodec and NpyRef from public API - Bump version to 2.0.0a17 Key features: - Lazy loading: inspect shape/dtype without downloading - NumPy integration via __array__ protocol - Safe bulk fetch: returns NpyRef objects, not arrays - Schema-addressed paths: {schema}/{table}/{pk}/{attr}.npy Co-Authored-By: Claude Opus 4.5 <[email protected]>
1 parent 2e79229 commit 8d7c92e

File tree

13 files changed

+1046
-147
lines changed

13 files changed

+1046
-147
lines changed

src/datajoint/__init__.py

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -47,16 +47,19 @@
4747
"MatStruct",
4848
# Codec API
4949
"Codec",
50+
"SchemaCodec",
5051
"list_codecs",
5152
"get_codec",
53+
"ObjectRef",
54+
"NpyRef",
55+
# Other
5256
"errors",
5357
"migrate",
5458
"DataJointError",
5559
"key",
5660
"key_hash",
5761
"logger",
5862
"cli",
59-
"ObjectRef",
6063
"ValidationResult",
6164
]
6265

@@ -70,6 +73,10 @@
7073
get_codec,
7174
list_codecs,
7275
)
76+
from .builtin_codecs import (
77+
SchemaCodec,
78+
NpyRef,
79+
)
7380
from .blob import MatCell, MatStruct
7481
from .connection import Connection, conn
7582
from .errors import DataJointError

0 commit comments

Comments
 (0)