-
Notifications
You must be signed in to change notification settings - Fork 22
Open
Description
Problem Statement:
Training Hub currently lacks a callback mechanism for extending training behavior at runtime. HuggingFace Transformers provides TrainerCallback that allows users to hook into training lifecycle events (epoch start/end, step end, evaluation, etc.). Training Hub should provide similar extensibility.
Without callbacks, users cannot inject custom logic during training without modifying Training Hub internals or relying on external file polling workarounds.
Desired API
A callback system similar to HuggingFace Transformers:
from training_hub.algorithms.sft import sft
from training_hub.callbacks import TrainingCallback
class MyCallback(TrainingCallback):
def on_step_end(self, args, state, control, **kwargs):
# Custom logic here
pass
sft(..., callbacks=[MyCallback()])
Use Cases
- Progress tracking and monitoring
- Integration with experiment tracking platforms (MLflow, Weights & Biases)
- Custom training behaviors (early stopping, dynamic learning rates)
- Real-time updates to external systems
Reference
HuggingFace Transformers TrainerCallback: https://huggingface.co/docs/transformers/main_classes/callback
astefanutti and robert-bell
Metadata
Metadata
Assignees
Labels
No labels