Skip to content

[Enhancement] Log the global gradient norms to tensorboard #95

@benbellheron

Description

@benbellheron

Adept uses global gradient norm clipping of 0.5 by default which could prevent learning (just slowing it down) if the gradients are high and the clipping occurs at every training step. At minimum we should log this to tensorboard so the user can decide/view for themselves.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions