Skip to content

Conversation

@LTMeyer
Copy link
Collaborator

@LTMeyer LTMeyer commented May 29, 2025

Closes #29

This PR adds the PytorchCodecHubMixin to be able to load and save codecs in the same model repo as AION.

Upload Codec Model to HF

Once the class inherits from this new Mixin, the following will upload the codec as a regular model, except that instead of having its own model repo it will now be stored under <model_repo>/codecs/<modality.name> of the model_repo.

codec.push_to_hub("polymathic-ai/aion-base")

Load Codec Model from HF

To load the model from HF, one has to specify the modality.

from aion.codecs import ScalarCodec
from aion.modalities import Dec

codec = ScalarCodec.from_pretrained("polymathic-ai/aion-base", modality=Dec)

ToDo

If the design seems good to you, it remains to:

  • Change the inheritance from PytorchHubMixin to PytorchCodecHubMixin
  • Check modality validity (addressed in Small Refactor #31)
  • Add small tests
  • Upload all the codecs on HF

Note that the question of different versions of AION is not addressed with this PR.

@LTMeyer LTMeyer requested a review from EiffL May 29, 2025 14:12
@LTMeyer LTMeyer changed the title WIP: PyTorchCodecHubMixin for Codec Models CodecPytorchHubMixin for Codec Models May 30, 2025
EiffL and others added 3 commits June 17, 2025 15:00
- Replace global state manipulation with thread-safe context manager
- Add comprehensive modality validation in from_pretrained method
- Fix save_pretrained method to handle modality parameter properly
- Simplify modality hierarchy by merging BaseModality and Modality classes
- Remove duplicate NewImageCodec class
- Improve error handling and type safety

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
Remove redundant Modality inheritance from all scalar modality classes.
Since Scalar already inherits from Modality, the dual inheritance
(Scalar, Modality) was unnecessary and has been simplified to (Scalar).

This affects all scalar modalities including:
- Legacy Survey flux measurements (G, R, I, Z, W1-W4)
- Shape parameters (R, E1, E2)
- HSC magnitudes and extinction values
- Gaia flux measurements and parallax
- Coordinate modalities (Ra, Dec)
- Spectral coefficients (GaiaXpBp, GaiaXpRp)

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
Copy link
Contributor

@EiffL EiffL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@EiffL EiffL merged commit fa38772 into main Jun 17, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Upload model to HF

3 participants