Major Changes! Modernize, add new features, adapt to the latest kpms version, pipeline redesign, new report schema

[MilagrosMarin]

element-moseq v1.0.0: Major Refactor with Breaking Changes

This pull request delivers a comprehensive refactor of the element-moseq project, focusing on environment management, configuration handling, and pipeline enhancements. Most notably, the code is updated to work with the latest version of keypoint-moseq, introduces new features that are incompatible with previous element-moseq versions, and modernizes the development and packaging setup.

Breaking Changes

• keypoint-moseq updated from pinned 0.4.8 to the latest upstream version (installed from source)
• Introduces five major breaking changes that are not backward-compatible with earlier element-moseq versions:
  • Rename kpms_reader functions to generate, load, and update kpms config files
  • Rename PCAPrep to PreProcessing table and add new attributes
  • Add feature to remove outlier keypoints in PreProcessing table with new outlier_scale_factor attribute in PCATask (only available in latest version of kpms)
  • Add sigmasq_loc feature in PreFit and FullFit to automatically estimate sigmasq_loc (prior controlling the centroid movement across frames) (only available in latest version of kpms)
  • Refactor PreProcessing table to use 3-part make function with new Video part table

These changes require database and pipeline alignment with the latest keypoint-moseq conventions.

New Features

• New report schema with comprehensive reporting capabilities
• New plotting module with refactored viz_utils for better organization
• Enhanced PreProcessing table with 3-part make function, new Video part table, and attributes video_duration, frame_rate, and average_frame_rate
• Support for both DLC config file extensions (config.yml and config.yaml)
• Updated model naming and folder structure with improved varchar lengths and primary attribute-based naming

Environment & Packaging Modernization

• Migrated from setup.py → pyproject.toml and conda_env.yml, aligning with modern Python packaging standards
• Simplified and consolidated environment definitions for easier reproducibility
• Updated devcontainer to Python 3.11 and upgraded dependencies
• Removed JAX dependencies from pyproject.toml
• Updated pre-commit hooks for improved linting and consistency

Configuration & Path Handling

• Fixed config functions with proper renaming to match keypoint-moseq logic
• Updated all usages (e.g., in moseq_infer.py, moseq_train.py, report.py) to ensure consistent, maintainable DataJoint config handling
• Improved path and directory handling using Path objects and robust existence checks
• Refactored moseq_infer paths and applied 3-part make function
• Updated folder naming logic to use string of combined primary attributes instead of datetime in PreFit and FullFit

Bug Fixes & Improvements

• Removed redundancy of variables in PreProcessing table
• Fixed deprecated datetime usage
• Fixed filename generation in Inference table
• Bugfix in Model imported foreign key
• Updated tutorial pipeline and documentation
• Enhanced docstrings across all modules
• Updated pipeline images to reflect new architecture

Impact Summary

• 135 total commits since August 2024
• 42 significant commits with features, fixes, and refactoring
• 5 breaking changes requiring user attention
• Major version bump from 0.3.2 to 1.0.0 due to breaking changes

[Copilot]

Pull Request Overview

Copilot reviewed 13 out of 17 changed files in this pull request and generated 4 comments.

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

The merge-base changed after approval.

The merge-base changed after approval.