Skip to content

Conversation

@harsh-sutariya
Copy link
Contributor

  • Implement AdaBelief optimizer following the NeurIPS 2020 paper
  • Add unit tests following MLX standard patterns
  • Update optimizer documentation to include AdaBelief

Fixes #2479

Proposed changes

  • Added AdaBelief class to python/mlx/optimizers/optimizers.py
  • Implemented with bias correction enabled by default (matching paper recommendations)
  • Supports decoupled weight decay
  • Added unit tests in python/tests/test_optimizers.py
  • Updated documentation in docs/src/python/optimizers/common_optimizers.rst

Checklist

  • I have read the CONTRIBUTING document
  • I have run pre-commit run --all-files to format my code / installed pre-commit prior to committing changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the necessary documentation (if needed)

- Implement AdaBelief optimizer following the NeurIPS 2020 paper
- Add unit tests following MLX standard patterns
- Update optimizer documentation to include AdaBelief

Fixes ml-explore#2479
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature Request: Add AdaBelief Optimizer

1 participant