feat: refactor main_ds.py

Introduce a new design for key components of main_ds.py. Namely splitting Model initialization, Accelerator initialization, Optimizer initialization, and Checkpoint saving initialization
into classes:

Model
Accelerator
Checkpointer

The Model class wraps the various AutoModel classes we support -- and aims to be a lightweight wrapper to help with usability of the library with different model types.
setup_optimizer resides within the model class and returns one of the optimizer types we support

The Accelerator class aims to both store commonly accessed variables associated with the accelerated model and abstract model/optimizer mutation away from the user who should only access our Model and Optimizer classes

The Checkpointer class introduces a unified approach to our various checkpointing techniques. A user can pass in their checkpointing style (full_state or hf_format), and the checkpointer, via checkpointer.checkpoint, will save the model using the selected method and other techniques (LoRA).

These classes are one of a few steps needed to "SDK-ify" the training library

Signed-off-by: Charlie Doern <cdoern@redhat.com>

Author: cdoern

src/instructlab/training/config.py

@@ -27,6 +27,20 @@ class DeepSpeedOffloadStrategy(Enum):
     NONE = None
 
 
+# public API
+class Optimizers(Enum):
+    ADAMW = "Adamw"
+    CPUAdam = "CPUAdam"
+    FusedAdam = "FusedAdam"
+
+
+# public API
+class ModelTypes(Enum):
+    LIGER = "Liger"
+    CAUSALLM = "Causallm"
+    DOLOMITE = "Dolomite"
+
+
 # public API
 class DistributedBackend(Enum):
     FSDP = "fsdp"