feat: virtual schema infrastructure and CI improvements

[dimitri-yatsenko]

Summary

This PR modernizes DataJoint's schema exploration capabilities with a complete virtual schema infrastructure overhaul and CLI improvements.

Virtual Schema Infrastructure (#1307)

New Schema Introspection API:

• Schema.get_table(name) - Direct table access with automatic tier prefix detection
• Schema.__getitem__ - Bracket notation: schema['TableName']
• Schema.__iter__ - Iterate over all tables in dependency order
• Schema.__contains__ - Check table existence: 'TableName' in schema

New Entry Points:

• dj.virtual_schema(schema_name) - Clean function to access existing database schemas
• dj.VirtualModule(alias, schema_name) - Create virtual modules with custom names

Internal Fixes:

• Fix gc.py to use get_table() instead of non-existent spawn_table()
• Handle tier prefixes (#, _, __) automatically in table lookups

CLI Improvements

The dj command-line interface now provides a proper interactive REPL for schema exploration:

Bug Fixes:

• Fix -h conflict: removed -h shorthand for --host (was overriding --help)
• Fix namespace issue: explicitly includes dj in REPL

Enhancements:

• Add module-level docstring with usage examples
• Improve function docstring to NumPy style
• Add error handling for invalid schema format (schema:alias validation)
• Improve banner with version and tab-completion hint

Usage:

# Start REPL with schemas loaded
dj -s my_lab:lab -s my_analysis:analysis

# In REPL
>>> lab.Subject.to_dicts()
>>> dj.Diagram(lab.schema)

Empty Insert Support (#1280)

Tables with all-default attributes now accept empty inserts:

@schema
class EventLog(dj.Manual):
    definition = """
    id: int auto_increment
    ---
    dt=CURRENT_TIMESTAMP : datetime
    """

EventLog().insert1({})  # Now works!

CI/Tooling (#1271)

• Use pixi for CI workflow with testcontainers
• Add mypy type checking to pre-commit
• Add unit tests to pre-commit hooks
• Update pre-commit to reference ruff (not flake8)
• Fix GitHub Actions labeler to preserve manual labels

Other Fixes

• Fix dj.Top to preserve ORDER BY in results (Test fails in relational_operand due to test code, not functionality. #1205)
• Use MySQL server time consistently for job scheduling
• Update repository keywords for 2.0

Breaking Changes

• Removed create_virtual_module - use dj.virtual_schema() or dj.VirtualModule() instead
• Removed specs/ folder (migrated to datajoint-docs)

Test Plan

• Virtual schema tests added (test_virtual_module.py)
• Schema introspection tests: get_table, __getitem__, __iter__, __contains__
• Empty insert tests added (test_insert.py::TestEmptyInsert)
• dj.Top ordering test updated
• CI passes

Closes

• Modernize virtual schema infrastructure #1307 (Modernize virtual schema infrastructure)
• IMPR: Error specificity on empty insert for table with default values #1280 (Empty insert for tables with all defaults)
• IMPR: Modernize pre-commit #1271 (Modernize pre-commit)
• Can't pass super.delete kwargs to Part.delete #1276 (Part.delete kwargs pass-through)
• Test fails in relational_operand due to test code, not functionality. #1205 (Test fails in relational_operand)

🤖 Generated with Claude Code