Skip to content

Commit 7869a94

Browse files
committed
fix: auto-generate lora_config.json in training script
This fix addresses the missing lora_config.json files in LoRA intent classifier models, which caused Go tests to skip and model routing to fail. Issue discovered by @yehudit1987 while investigating skipped classifier tests. Changes: - Updated merge_lora_adapter_to_full_model() in ft_linear_lora.py to automatically create lora_config.json when merging LoRA adapters The lora_config.json file contains LoRA metadata (rank, alpha, dropout, target_modules) required by the Rust router to detect LoRA models and route them to the correct inference path. The file will be manually uploaded to HuggingFace for existing models, and future training runs will generate it automatically. Fixes #627 Signed-off-by: Yossi Ovadia <[email protected]>
1 parent 33306f4 commit 7869a94

File tree

1 file changed

+21
-0
lines changed

1 file changed

+21
-0
lines changed

src/training/training_lora/classifier_model_fine_tuning_lora/ft_linear_lora.py

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -659,6 +659,27 @@ def merge_lora_adapter_to_full_model(
659659
)
660660
logger.info("Created category_mapping.json")
661661

662+
# Create lora_config.json for Rust router detection
663+
# This file signals to the Rust router that this is a LoRA-trained model
664+
# and should be routed to the LoRA inference path
665+
logger.info("Creating lora_config.json for LoRA model detection...")
666+
lora_config = {
667+
"rank": 16, # LoRA rank (r) - matches training configuration
668+
"alpha": 32, # LoRA alpha scaling factor
669+
"dropout": 0.1, # LoRA dropout rate
670+
"target_modules": [
671+
"attention.self.query",
672+
"attention.self.value",
673+
"attention.output.dense",
674+
"intermediate.dense",
675+
"output.dense",
676+
],
677+
}
678+
lora_config_path = os.path.join(output_path, "lora_config.json")
679+
with open(lora_config_path, "w") as f:
680+
json.dump(lora_config, f)
681+
logger.info(f"Created {lora_config_path}")
682+
662683
logger.info("LoRA adapter merged successfully!")
663684

664685

0 commit comments

Comments
 (0)