Skip to content

Commit 1e9ccca

Browse files
committed
Fix calculation of "nearest integer" function (bug #67339)
The internal function x_nint in lo-mappers.h was being used to find the nearest integer value to a given input x, using the expression 'std::floor (x + 0.5)'. This expression was giving wrong results for x > 0.5 * flintmax, because the ratio 0.5 / x was becoming lower than eps at that point, and that in turn was causing incorrect rounding for odd numbers. In turn, it was causing problems with mod() and rem() with numbers near (but still lower than) flintmax where those functions were returning zero for mod (x-1, x) instead of x-1 itself. * lo-mappers.h (x_nint): Use std::round instead of handrolled code. * data.cc (Fmod, Frem): New BISTs for inputs near flintmax().
1 parent c67f93e commit 1e9ccca

File tree

2 files changed

+16
-2
lines changed

2 files changed

+16
-2
lines changed

libinterp/corefcn/data.cc

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -756,6 +756,13 @@ periodic, @code{mod} is a better choice.
756756
757757
%!assert <*42627> (rem (0.94, 0.01), 0.0)
758758
759+
## Test rem (x-1, x) for x close to flintmax. Should return x-1 and not zero.
760+
%!test <*67339>
761+
%! x = flintmax - (10:-1:1);
762+
%! assert (rem (x-1, x), x-1);
763+
%! x = flintmax ("single") - (10:-1:1);
764+
%! assert (rem (x-1, x), x-1);
765+
759766
%!error rem (uint (8), int8 (5))
760767
%!error rem (uint8 ([1, 2]), uint8 ([3, 4, 5]))
761768
%!error rem ()
@@ -958,6 +965,13 @@ negative numbers or when the values are periodic.
958965
%!assert <*54602> (mod (int8 (-125), int8 (0)), int8 (-125))
959966
%!assert <*54602> (mod (int8 (0), int8 (-25)), int8 (0))
960967
968+
## Test mod (x-1, x) for x close to flintmax. Should return x-1 and not zero.
969+
%!test <*67339>
970+
%! x = flintmax - (10:-1:1);
971+
%! assert (mod (x-1, x), x-1);
972+
%! x = flintmax ("single") - (10:-1:1);
973+
%! assert (mod (x-1, x), x-1);
974+
961975
*/
962976

963977
#define DATA_REDUCTION(FCN) \

liboctave/numeric/lo-mappers.h

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -274,13 +274,13 @@ x_nint (T x)
274274
template <>
275275
inline double x_nint (double x)
276276
{
277-
return (isfinite (x) ? std::floor (x + 0.5) : x);
277+
return std::round (x);
278278
}
279279

280280
template <>
281281
inline float x_nint (float x)
282282
{
283-
return (isfinite (x) ? std::floor (x + 0.5f) : x);
283+
return std::round (x);
284284
}
285285

286286
extern OCTAVE_API octave_idx_type nint_big (double x);

0 commit comments

Comments
 (0)