Version 0.5.0 with numpy fails on Decimal values

[emakarov]

UTM version:

pip freeze | grep utm
utm==0.5.0

Numpy version:

pip freeze | grep numpy
numpy==1.16.4

(tested on some other numpy versions below 1.16.4)

Code:

from decimal import *
import utm
lat, lon = 1.286834229215, 103.796852963067
utm.from_latlon(Decimal(lat), Decimal(lon))

fails with such traceback:

~/dev/envs/p3/lib/python3.6/site-packages/utm/conversion.py in from_latlon(latitude, longitude, force_zone_number, force_zone_letter)
    187        .. _[1]: http://www.jaworski.ca/utmzones.htm
    188     """
--> 189     if not in_bounds(latitude, -80.0, 84.0):
    190         raise OutOfRangeError('latitude out of range (must be between 80 deg S and 84 deg N)')
    191     if not in_bounds(longitude, -180.0, 180.0):

~/dev/envs/p3/lib/python3.6/site-packages/utm/conversion.py in in_bounds(x, lower, upper, upper_strict)
     47         return lower <= x < upper
     48     elif use_numpy:
---> 49         return lower <= mathlib.min(x) and mathlib.max(x) <= upper
     50     return lower <= x <= upper
     51

~/dev/envs/p3/lib/python3.6/site-packages/numpy/core/fromnumeric.py in amin(a, axis, out, keepdims, initial)
   2616     """
   2617     return _wrapreduction(a, np.minimum, 'min', axis, None, out, keepdims=keepdims,
-> 2618                           initial=initial)
   2619
   2620

~/dev/envs/p3/lib/python3.6/site-packages/numpy/core/fromnumeric.py in _wrapreduction(obj, ufunc, method, axis, dtype, out, **kwargs)
     82                 return reduction(axis=axis, dtype=dtype, out=out, **passkwargs)
     83             else:
---> 84                 return reduction(axis=axis, out=out, **passkwargs)
     85
     86     return ufunc.reduce(obj, axis, dtype, out, **passkwargs)

TypeError: Required argument 'other' (pos 1) not found

[TheBB]

We're going to have to wrap a few of the math functions if we want to support Decimals in numpy-mode I think. Few numpy functions understand them, even when used as straight-up scalars, including some we need: sin, cos, radians, degrees, max and min. Interestingly sqrt is fine with them.

I'm not sure if numpy would consider this a bug.

[emakarov]

I'm not sure about numpy strategy about Decimal support in future.
The issue was found working with django project where lat/lon values are kept in decimals in database.

I see two options:

1. Consider this not a bug and raise specific Exception about attribute type (I think it's less preferred variant)
2. Cast Decimals to float explicitly in the code of utm

[emakarov]

the situation is also following:

if you call numpy.max with single float argument, it works
if you call numpy.max with array of floats, it works
if you call numpy.max with array of Decimals, it works
if you call numpy.max with single Decimal, it fails

[bartvanandel]

Just as a reminder, this is still unsupported by (our required version of) numpy and therefore by utm (both the current release, 0.7.0, and current git).

[bartvanandel]

I investigated a bit more, and it looks like the only viable solution for now would indeed be to either raise an error or cast to float (or np.float64, which numpy already casts to internally).

Actually, you can't use UTM.to_utm with Decimal input at all. You can use UTM.from_utm, but only because calls to trigonometric functions will return float output. Multiplying a Decimal by a float, or likewise, dividing them, is unsupported.

The problem is not just the min and max functions (which indeed do work for lists of values), but more importantly the trigonometric functions, which are simply not implemented for Decimals, neither as single values nor as lists of values. Actually, UTM.from_latlon only works with Decimal input because the built-in trigonometric functions return float output for Decimal input. UTM.to_latlon tries to divide, which isn't supported for mixed Decimal / float parameters (and neither is multiplying, adding, subtracting etc).

In practice it's quite pointless to use more than say 7 decimals in UTM coordinates, which is already enough for "pointing to Waldo on a page".

I was about to add a guard / conversion for Decimal input, but it seems this is such an edge case that we'd better just document our lack of support and not sacrifice performance.

For those interested, conversion is quite easy. Essentially, my patch would contain the following code:

from decimal import Decimal
try:
    import numpy as np
    mathlib = np
    use_numpy = True
except:
    use_numpy = False


# We don't support Decimal input. It isn't required for accurate representation
# of UTM coordinates, and it would complicate our calculations. Numpy doesn't
# support it at all for anything but the simplest functions. However, rather
# than simply crashing, we'll simply downcast Decimal input to float.
if use_numpy:
    def cast_decimals(value):
        if isinstance(value, Decimal):
            return np.float64(value)
        elif isinstance(value, np.ndarray) and value.size > 0 and isinstance(value.flat[0], Decimal):
            return value.astype(np.float64)
        elif isinstance(value, (list, tuple)) and len(value) > 0 and isinstance(value[0], Decimal):
            return type(value)(float(x) for x in value)
        return value
else:
    def cast_decimals(value):
        if isinstance(value, Decimal):
            return float(value)
        return value


# Inside `to_latlon`:
easting = cast_decimals(easting)
northing = cast_decimals(northing)


# Inside `from_latlon`:
latitude = cast_decimals(latitude)
longitude = cast_decimals(longitude)

If there is enough interest, and no-one objects to adding a tiny bit of a performance hit for this (IMHO) edge case, I can still add it of course. I have a test case ready just in case. See this branch.

@Turbo87 what's your opinion on this?