Add loading of custom MATLAB datetime structures

[skjerns]

With the help of our glorious AI overlords Gemini and Claude I was able to reverse-engineer how datetimes are stored in MATLAB.

[gemini-code-assist]

Summary of Changes

Hello @skjerns, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the mat73 library's capability to correctly interpret and load MATLAB datetime structures into native Python datetime objects. By supporting both the newer MCOS object encoding and the traditional numeric datenum format, it ensures broader compatibility and accurate data representation for users working with MATLAB .mat files containing temporal data. The changes provide a robust and reliable mechanism for datetime conversion, addressing potential issues with different storage methods and edge cases like NaT values.

Highlights

• MATLAB Datetime Decoding: Introduced comprehensive support for loading MATLAB datetime objects, handling both modern MCOS-encoded structures and legacy numeric date arrays (datenums).
• MCOS Object Handling: Implemented logic to navigate the HDF5 MCOS subsystem to extract Unix timestamps in milliseconds for MCOS-encoded datetimes and convert them to Python datetime objects.
• Legacy Datetime Conversion: Added conversion for MATLAB's numeric datenums (days since January 0, 0000) to Python datetime objects, including handling of NaT (Not-a-Time) values and dates before 0001-01-01.
• Extensive Test Coverage: New MATLAB script (create_mat.m) and Python test cases (test_mat73.py) were added to validate the loading of various datetime formats, including scalars, vectors, matrices, specific times with milliseconds, NaT values, timezones, and datetimes embedded within structs and cell arrays.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• mat73/core.py
  • Added datetime and timedelta imports from the datetime module.
  • Implemented a new elif MATLAB_class == 'datetime' block within the convert_mat method to handle MATLAB datetime objects.
  • Developed logic to differentiate between MCOS-encoded datetimes (identified by MATLAB_object_decode=3 attribute) and legacy numeric datenums.
  • For MCOS datetimes, the code now navigates the HDF5 file's MCOS subsystem, extracts 0-based reference indices, and converts millisecond Unix timestamps to Python datetime objects, preserving original array shapes and handling NaT values.
  • For legacy datenums, the code converts MATLAB's serial date numbers (days since 0000-00-00) to Python datetime objects, applying a MATLAB_TO_PYTHON_OFFSET and handling NaT and dates prior to 0001-01-01 with warnings.
• tests/create_mat.m
  • Added a new section to generate testfile_datetime.mat.
  • This new test file includes a comprehensive set of MATLAB datetime variables: scalar, row vector, column vector, 2x2 matrix, specific time with milliseconds, NaT, array with NaT, datetime with timezone, past date (1500-01-01), future date (2500-01-01), datetimes within a struct, and datetimes within cell arrays (both row and column oriented).
• tests/test_mat73.py
  • Added datetime import from the datetime module.
  • Introduced a new test method test_datetime_loading to verify the correct parsing of MATLAB datetime objects.
  • Included a helper function assert_datetime_equal for precise comparison of datetime objects.
  • The test method loads testfile_datetime.mat and asserts the correctness of all datetime variables, checking for type, year, month, day, hour, minute, second, and microsecond, as well as proper handling of NaT and array structures.

Activity

• The author, skjerns, initiated this pull request to address the loading of custom MATLAB datetime structures.
• The PR description indicates that AI tools (Gemini and Claude) were utilized to reverse-engineer MATLAB's datetime storage mechanisms, which informed the implementation.
• The changes involve significant code additions to the core parsing logic and the creation of new test data and test cases to ensure functionality and correctness.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces robust support for loading MATLAB datetime objects, covering both MCOS-encoded and legacy numeric date array formats. The implementation includes comprehensive logic for handling various datetime scenarios, such as scalar values, arrays, NaT (Not-a-Time), and timezone considerations. The accompanying test cases are thorough and validate the new functionality effectively across different MATLAB data structures. The changes significantly enhance the mat73 library's ability to interpret complex MATLAB data types.

[gemini-code-assist]

The MATLAB_TO_PYTHON_OFFSET is a constant value that is currently defined inside the convert_mat method. For better maintainability and to avoid re-assignment on every function call, it's good practice to define such constants at the module level (e.g., near other imports or global variables).

Suggested change

	MATLAB_TO_PYTHON_OFFSET = 366
	# MATLAB uses days since January 0, 0000, Python uses days since January 1, 0001
	MATLAB_TO_PYTHON_OFFSET = 366