enable powers of 2 cast in float8 rowwise_with_gw_hp recipe (#2677)

vkuzo · web-flow · commit 2f79364a150e · 2025-08-06T10:12:36.000-04:00
Summary:

This should have been enabled from the time we added the powers of 2
scaling, fixing.

Test Plan:

```
with-proxy
CONFIG_FILE="./torchtitan/models/llama3/train_configs/llama3_8b.toml"
./run_train.sh --model.print_after_conversion --model.converters float8
--training.compile --float8.recipe_name rowwise_with_gw_hp
```

Reviewers:

Subscribers:

Tasks:

Tags:
diff --git a/torchao/float8/config.py b/torchao/float8/config.py
@@ -333,6 +333,7 @@ def from_recipe_name(
                 cast_config_input_for_grad_weight=cc_i_gw,
                 cast_config_weight_for_grad_input=cc_w_gi,
                 cast_config_grad_output_for_grad_weight=cc_go_gw,
+                round_scales_to_power_of_2=True,
             )
 
         else:

Original file line number	Diff line number	Diff line change
`@@ -333,6 +333,7 @@ def from_recipe_name(`
`333`	`333`	`cast_config_input_for_grad_weight=cc_i_gw,`
`334`	`334`	`cast_config_weight_for_grad_input=cc_w_gi,`
`335`	`335`	`cast_config_grad_output_for_grad_weight=cc_go_gw,`
	`336`	`+ round_scales_to_power_of_2=True,`
`336`	`337`	`)`
`337`	`338`
`338`	`339`	`else:`