Skip to content

Commit 0b34362

Browse files
authored
Fix reduction param validation for flce (#879)
## Summary <!--- This is a required section; please describe the main purpose of this proposed code change. ---> This is a fix for Issue #872. @shimizust wants me to submit this fix. <!--- ## Details This is an optional section; is there anything specific that reviewers should be aware of? ---> @shimizust Could you have a look at the fix? ## Testing Done <!--- This is a required section; please describe how this change was tested. ---> <!-- Replace BLANK with your device type. For example, A100-80G-PCIe Complete the following tasks before sending your PR, and replace `[ ]` with `[x]` to indicate you have done them. --> - Hardware Type: <BLANK> - [ ] run `make test` to ensure correctness - [ ] run `make checkstyle` to ensure code style - [ ] run `make test-convergence` to ensure convergence
1 parent 454e3d2 commit 0b34362

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

src/liger_kernel/transformers/fused_linear_cross_entropy.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,8 @@ def __init__(
2525
assert reduction in {
2626
"mean",
2727
"sum",
28-
}, f"reduction must be 'mean' or 'sum'. Got: {reduction}"
28+
"none",
29+
}, f"reduction must be 'mean' or 'sum' or 'none'. Got: {reduction}"
2930
assert softcap is None or softcap > 0, f"softcap must greater than 0.0 or None. Got: {softcap}"
3031
self.ce_weight = ce_weight
3132
self.ignore_index = ignore_index

0 commit comments

Comments
 (0)