Validation is incorrectly run on resume

[PiotrDabkowski]

Bug description

When val_check_interval is used, and the model is resumed the validation is run immediately after resume, wasting resources (causing ckpt save, and whole validation run).

#18110

What version are you seeing the problem on?

v2.4

How to reproduce the bug

No response

Error messages and logs

No response

Environment

No response

More info

No response

[PiotrDabkowski]

This has been reported once and fixed. But the regression has been introduced in 1.6.0 #11504

[ryxli]

Was fixed previously with (#11552):

        # while restarting with no fault-tolerant, batch_progress.current.ready is -1
        if batch_idx == -1:
            return False

batch_idx was removed some time back, now should the logic be?

        if self.restarting:
            return False

[PiotrDabkowski]

@lantiga this is a really problematic issue, just a completely bugged experience, would be nice to get it fixed asap.

[Rilwan-Adewoyin]

Was fixed previously with (#11552):

        # while restarting with no fault-tolerant, batch_progress.current.ready is -1
        if batch_idx == -1:
            return False

batch_idx was removed some time back, now should the logic be?

        if self.restarting:
            return False

Could you provide the file where you suggest this change needs to be made?