DataJoint 2.0 Release Planning

[dimitri-yatsenko]

As we approach the final stages of development, we are excited to announce DataJoint 2.0, our first major release designed to align with the official DataJoint Specs 2.0

Target release date: Oct 2025.

---

The centerpiece of this release is the introduction of Object-Augmented Schemas. We are also delivering a fully modernized framework for project configuration, data handling, and new scientific datatype support.

**Compatibility Note:*While we aim for data compatibility (meaning no data migration should be required), this release will introduce several backward-incompatible API features. Please plan to update your code to leverage the new functionalities.

---

Object-Augmented Schemas

DataJoint 2.0 fundamentally revises how relational tables are combined with large-scale scientific data objects. This new paradigm allows for more flexible and efficient handling of complex data types directly within the data pipeline.

• Core Implementation: Foundational support for Object-Augmented Schemas.
• Custom Data Type Plugins: An extensible architecture for user-defined data types.
  • <dj-blob>: A plugin for backward-compatible DataJoint blobs.
  • <dj-zarr>: A plugin for genera Zarr 3.0 array support
  • <dj-photon>: plugin for handling compressed, photon-limited movies.

Additional Features & Roadmap

Version 2.0 (Target: Oct 2025)

• Project Configuration Management
  • Unified management of secrets, config files, and environment variables.
  • A modern plugin architecture of enhanced extensibility
• Schema Definition Syntax
  • Scientific datatype aliases: float32, float64, uint8, int8, uint16, int16, uint32, int32, uint64, int64.
• Data Export & Import
  • Diagram-based data restriction for queries
  • Diagram-based export and data publishing workflows.
  • Functionality to restore a database state from an export.

Version 2.1 (Post 2.0 Release)

The following features are planned for a subsequent release shortly after 2.0, with a possibility of being included in the main release.

• PostgreSQL Support
• Native JSON Datatype Support