Skip to content

Conversation

@bhimrazy
Copy link
Collaborator

@bhimrazy bhimrazy commented Oct 16, 2025

What does this PR do?

Adds a warning to the 16-bit precision documentation about float16's inability to represent values smaller than ~6e-5, which can cause NaN during training.

Fixes #20619

Before submitting
  • Was this discussed/agreed via a GitHub issue? (not for typos and docs)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes? (if necessary)
  • Did you write any new necessary tests? (not for typos and docs)
  • Did you verify new and existing tests pass locally with your changes?
  • Did you list all the breaking changes introduced by this pull request?
  • Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

Reviewer checklist
  • Is this pull request ready for review? (if not, please submit in draft mode)
  • Check that all items from Before submitting are resolved
  • Make sure the title is self-explanatory and the description concisely explains the PR
  • Add labels and milestones (and optionally projects) to the PR so it can be classified

📚 Documentation preview 📚: https://pytorch-lightning--21295.org.readthedocs.build/en/21295/

@bhimrazy bhimrazy self-assigned this Oct 16, 2025
@github-actions github-actions bot added docs Documentation related pl Generic label for PyTorch Lightning package labels Oct 16, 2025
@SkafteNicki SkafteNicki added the precision: half Half-precision label Oct 20, 2025
@bhimrazy bhimrazy merged commit 56204d8 into Lightning-AI:master Oct 21, 2025
23 checks passed
@bhimrazy bhimrazy deleted the docs/fp16-precision-warning branch October 21, 2025 12:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation related pl Generic label for PyTorch Lightning package precision: half Half-precision

Projects

None yet

Development

Successfully merging this pull request may close these issues.

16 bit precision in Trainer leading to NaN

4 participants