Skip to content

Add nan_to_num helper #796

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 7, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 60 additions & 0 deletions pytensor/tensor/math.py
Original file line number Diff line number Diff line change
Expand Up @@ -3043,6 +3043,65 @@ def vectorize_node_dot_to_matmul(op, node, batched_x, batched_y):
return vectorize_node_fallback(op, node, batched_x, batched_y)


def nan_to_num(x, nan=0.0, posinf=None, neginf=None):
"""
Replace NaN with zero and infinity with large finite numbers (default
behaviour) or with the numbers defined by the user using the `nan`,
`posinf` and/or `neginf` keywords.

NaN is replaced by zero or by the user defined value in
`nan` keyword, infinity is replaced by the largest finite floating point
values representable by ``x.dtype`` or by the user defined value in
`posinf` keyword and -infinity is replaced by the most negative finite
floating point values representable by ``x.dtype`` or by the user defined
value in `neginf` keyword.

Parameters
----------
x : symbolic tensor
Input array.
nan
The value to replace NaN's with in the tensor (default = 0).
posinf
The value to replace +INF with in the tensor (default max
in range representable by ``x.dtype``).
neginf
The value to replace -INF with in the tensor (default min
in range representable by ``x.dtype``).

Returns
-------
out
The tensor with NaN's, +INF, and -INF replaced with the
specified and/or default substitutions.
"""
# Replace NaN's with nan keyword
is_nan = isnan(x)
is_pos_inf = eq(x, np.inf)
is_neg_inf = eq(x, -np.inf)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can also add the numpy helpers for isneginf, isposinf for users (and use them here)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created Isposinf and Isneginf with the very same purpose here:
Dhruvanshu-Joshi@829309b

How can we use np.isposinf(x) directly in helper functions without creating an op when x is but a container?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't, you use the pt.isponsinf helper which will have this exact code inside pt.eq(pt.as_tensor(x), np.inf).


if not any(is_nan) and not any(is_pos_inf) and not any(is_neg_inf):
return

x = switch(is_nan, nan, x)

# Get max and min values representable by x.dtype
maxf = posinf
minf = neginf

# Specify the value to replace +INF and -INF with
if maxf is None:
maxf = np.finfo(x.real.dtype).max
if minf is None:
minf = np.finfo(x.real.dtype).min

# Replace +INF and -INF values
x = switch(is_pos_inf, maxf, x)
x = switch(is_neg_inf, minf, x)

return x


# NumPy logical aliases
square = sqr

Expand Down Expand Up @@ -3199,4 +3258,5 @@ def vectorize_node_dot_to_matmul(op, node, batched_x, batched_y):
"logaddexp",
"logsumexp",
"hyp2f1",
"nan_to_num",
]
29 changes: 29 additions & 0 deletions tests/tensor/test_math.py
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,7 @@
minimum,
mod,
mul,
nan_to_num,
neg,
neq,
outer,
Expand Down Expand Up @@ -3641,3 +3642,31 @@ def test_grad_n_undefined(self):
n = scalar(dtype="int64")
with pytest.raises(NullTypeGradError):
grad(polygamma(n, 0.5), wrt=n)


@pytest.mark.parametrize(
["nan", "posinf", "neginf"],
[(0, None, None), (0, 0, 0), (0, None, 1000), (3, 1, -1)],
)
def test_nan_to_num(nan, posinf, neginf):
x = tensor(shape=(7,))

out = nan_to_num(x, nan, posinf, neginf)

f = function(
[x],
nan_to_num(x, nan, posinf, neginf),
on_unused_input="warn",
allow_input_downcast=True,
)

y = np.array([1, 2, np.nan, np.inf, -np.inf, 3, 4])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would solve the failing float32 test without having to downcast the input

Suggested change
y = np.array([1, 2, np.nan, np.inf, -np.inf, 3, 4])
y = np.array([1, 2, np.nan, np.inf, -np.inf, 3, 4]).astype(x.dtype)

out = f(y)

posinf = np.finfo(x.real.dtype).max if posinf is None else posinf
neginf = np.finfo(x.real.dtype).min if neginf is None else neginf

np.testing.assert_allclose(
out,
np.nan_to_num(y, nan=nan, posinf=posinf, neginf=neginf),
)