Skip to content

Commit 43383eb

Browse files
committed
Fix potential NaN condition when applying repetition penalty
1 parent a5caa42 commit 43383eb

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

exllamav3/exllamav3_ext/generator/rep_pen.cu

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,8 @@ void apply_rep_pens_kernel
7575

7676
float w = v > 0.0f ? v / rep_p : v * rep_p;
7777
float f = factors[i] + 1e-30;
78-
float o = v * (1.0f - f) + w * f;
78+
float f1 = (1.0f - f) + 1e-30;
79+
float o = v * f1 + w * f;
7980
out_logits[i + range_min] = o;
8081
}
8182
}

0 commit comments

Comments
 (0)