Update base for Update on "CI test for lora"

Add CI test for LoRA: note, not program-data separated yet.

Differential Revision: [D78751767](https://our.internmc.facebook.com/intern/diff/D78751767/)

[ghstack-poisoned]

Author: lucylq

requirements-examples.txt

@@ -3,5 +3,5 @@
 datasets == 3.6.0 # 4.0.0 deprecates trust_remote_code and load scripts. For now pin to 3.6.0
 timm == 1.0.7
 torchsr == 1.0.4
-torchtune >= 0.7.0
+torchtune >= 0.6.1
 transformers == 4.53.1