Skip to content

Commit b7e93bf

Browse files
authored
Better rounding of BigMath.atan(nearly_one, prec) (#469)
1 parent cef76eb commit b7e93bf

File tree

2 files changed

+2
-1
lines changed

2 files changed

+2
-1
lines changed

lib/bigdecimal/math.rb

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -286,7 +286,7 @@ def atan(x, prec)
286286
pi = PI(n)
287287
x = -x if neg = x < 0
288288
return pi.div(neg ? -2 : 2, prec) if x.infinite?
289-
return pi.div(neg ? -4 : 4, prec) if x.round(prec) == 1
289+
return pi.div(neg ? -4 : 4, prec) if x.round(n) == 1
290290
x = BigDecimal("1").div(x, n) if inv = x > 1
291291
x = (-1 + sqrt(1 + x.mult(x, n), n)).div(x, n) if dbl = x > 0.5
292292
y = x

test/bigdecimal/test_bigmath.rb

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -278,6 +278,7 @@ def test_atan
278278
assert_converge_in_precision {|n| atan(BigDecimal("2"), n)}
279279
assert_converge_in_precision {|n| atan(BigDecimal("1e-30"), n)}
280280
assert_converge_in_precision {|n| atan(BigDecimal("1e30"), n)}
281+
assert_equal(BigDecimal(0.78), BigMath.atan(0.996, 2))
281282
end
282283

283284
def test_atan2

0 commit comments

Comments
 (0)