Skip to content

ETA is grossly over-estimated.Β #77

@JenkinsTR

Description

@JenkinsTR

I think your ETA calculation is off by a factor of 100...

Through both my own manual calculations and visually inspecting the ETA while training, it seems that the decimal separator is placed in the wrong spot.

Example:
23-07-26 02:16:57.959 - INFO: <epoch: 33, iter: 167,900, lr:6.250e-06, t:31.6040s, td:0.0001s, eta:281.8024h> pix-l1: 4.2061e-04 fea-vgg19-l1: 7.0643e-01 l_g_gan: 1.4697e-02 l_d_real: 5.6621e-02 l_d_fake: 5.6456e-02 D_real: 4.0469e+01 D_fake: 3.7594e+01

Here we see that the ETA is 281.8024h(rs), when in fact it is much closer to 2.81hrs. This is consistent with every training run I've done so far, no matter how big or small the ETA actually is.

Even though I've gotten used to it now, I thought I might raise an issue here and let you know πŸ‘

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions