Skip to content

Commit 3a7b9ca

Browse files
committed
Fix Integer.sqrt to never exceed actual value
`Integer.sqrt` uses `sqrt(3)` from libm for small values. This method must return a value less than or equal to the actual integer square root, but libm's sqrt does not always guarantee that. This change corrects that by decrementing the result if necessary. Fixes [Bug #21217]
1 parent e258899 commit 3a7b9ca

File tree

2 files changed

+9
-1
lines changed

2 files changed

+9
-1
lines changed

numeric.c

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5978,7 +5978,11 @@ prefix##_isqrt(argtype n) \
59785978
while ((t = n/x) < (argtype)x) x = (rettype)((x + t) >> 1); \
59795979
return x; \
59805980
} \
5981-
return (rettype)sqrt(argtype##_TO_DOUBLE(n)); \
5981+
rettype x = (rettype)sqrt(argtype##_TO_DOUBLE(n)); \
5982+
/* libm sqrt may returns a larger approximation than actual. */ \
5983+
/* Our isqrt always returns a smaller approximation. */ \
5984+
if (x * x > n) x--; \
5985+
return x; \
59825986
}
59835987

59845988
#if SIZEOF_LONG*CHAR_BIT > DBL_MANT_DIG

test/ruby/test_integer.rb

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -708,6 +708,10 @@ def test_square_root
708708
assert_equal(x, Integer.sqrt(x ** 2), "[ruby-core:95453]")
709709
end
710710

711+
def test_bug_21217
712+
assert_equal(0x10000 * 2**10, Integer.sqrt(0x100000008 * 2**20))
713+
end
714+
711715
def test_fdiv
712716
assert_equal(1.0, 1.fdiv(1))
713717
assert_equal(0.5, 1.fdiv(2))

0 commit comments

Comments
 (0)