Pre-Forward and Post-Forward Callbacks?

[eldarkurtic]

Hi everyone. Does this library support adding calls to custom functions on specific points of the training loop? For example, calling a custom function before each forward pass and/or after each forward pass.

[NanoCode012]

Hello, since we wrap transformers Trainer. You can create a callback and add it to the trainer

Here are the possible hook options on_step_begin: https://huggingface.co/docs/transformers/main_classes/callback#transformers.TrainerCallback.on_step_begin

You can see the callbacks we have in the repo here: src/axolotl/utils/callbacks/__init__.py