Skip to content

Commit 51eb41a

Browse files
JonSnow1807pytorchmergebot
authored andcommitted
Enable dynamic shapes for foreach operations by default (pytorch#158985)
## Summary This PR changes the default value of `combo_kernel_foreach_dynamic_shapes` from `False` to `True` in `torch/_inductor/config.py`. ## Context The `combo_kernel_foreach_dynamic_shapes` configuration was introduced in PR pytorch#134477 (August 2024) to support dynamic shapes for foreach and combo kernels. It was initially disabled by default as a conservative approach to avoid disrupting production workflows. ## Why This Change? After several months of the feature being available and stable, it's time to enable it by default. This improves the user experience for developers using `torch.compile(dynamic=True)` with foreach operations. ### Current behavior: - Users must manually discover and enable `combo_kernel_foreach_dynamic_shapes` - Without this flag, foreach operations may fail with dynamic shapes - This creates friction and confusion ### With this change: - Foreach operations work seamlessly with dynamic compilation - No manual configuration needed - Better "it just works" experience ## Testing Extensive testing was performed with PyTorch 2.5.0+ and 2.7.1: - ✅ Various tensor sizes (8, 16, 32, 64, 128) - ✅ Multiple tensors in operations (tested up to 20) - ✅ Nested foreach operations - ✅ Mixed operations (foreach + standard operations) - ✅ Both CPU and CUDA devices - ✅ Symbolic shapes with dynamic compilation ## Impact Assessment - **Performance**: No impact - this only affects compilation behavior - **Backward Compatibility**: Fully maintained - users can still set to `False` - **Risk**: Minimal - feature has been stable since August 2024 ## References - Original implementation: PR pytorch#134477 by @qchip - This completes the feature rollout by making it available by default Pull Request resolved: pytorch#158985 Approved by: https://github.com/jansel, https://github.com/mlazos
1 parent ede6186 commit 51eb41a

File tree

2 files changed

+2
-1
lines changed

2 files changed

+2
-1
lines changed

test/inductor/test_foreach.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -645,6 +645,7 @@ def fn(a0, a1, b0, b1):
645645
@bin_ops
646646
@torch._dynamo.config.patch("automatic_dynamic_shapes", False)
647647
@torch._dynamo.config.patch("assume_static_by_default", False)
648+
@torch._inductor.config.patch("combo_kernel_foreach_dynamic_shapes", False)
648649
def test_dynamic_shapes_fallback(self, op):
649650
def fn(a0, a1, b0, b1):
650651
return op([a0, a1], [b0, b1])

torch/_inductor/config.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -677,7 +677,7 @@ def use_autoheuristic(name: str) -> bool:
677677
# for all except for foreach, 2 - enable for all
678678
combo_kernel_allow_mixed_sizes = 1
679679
# Enable dynamic shapes for foreach kernels
680-
combo_kernel_foreach_dynamic_shapes = False
680+
combo_kernel_foreach_dynamic_shapes = True
681681

682682
# constant folding on the joint graph
683683
joint_graph_constant_folding = True

0 commit comments

Comments
 (0)