How to save the outputs during training?

[liudan193]

Could you please explain how to save the outputs of the training and testing sets during the model training process, along with their log probabilities?

[NanoCode012]

Hey, this will require some internal adjustments. Since we wrap transformers, you could create a callback (on epoch end or on eval up to you)

There are two ways:

1. Place it to https://github.com/axolotl-ai-cloud/axolotl/blob/main/src/axolotl/utils/callbacks/__init__.py and include it within the trainer you're using

   axolotl/src/axolotl/core/builders/causal.py

   Line 58 in a5946ff

   def get_callbacks(self):

2. Place it within the integrations folder by creating an integration for add_callbacks_post_trainer then setting it as a plugin within the yaml https://github.com/axolotl-ai-cloud/axolotl/blob/main/src/axolotl/integrations/base.py#L240

I would recommend the first if you're just hacking around.