Elementwise exponentiation in LazyTensor

[nathanohara]

Hi all,

I'm using GPyTorch kernels in an application where I want to take 1/K for a given covariance matrix.

I've looked through the LazyTensor operations and I'm not sure there's a clear way to do this without converting to Tensor. Right now I'm doing something like this:

def invert_kernel_matrix(K: gpytorch.lazy.LazyTensor) -> torch.Tensor:
    """
    Return a matrix representing the elementwise multiplicative inverse of a kernel matrix K
    from GPyTorch represented as LazyTensor
    """
    return 1.0 / (K.evaluate() + 1e-6)

In my application this is fine, but interested in if there's a way to do this without converting to Tensor since the "inverted" matrix should also be able to be represented as LazyTensor. What's the most efficient way to perform ^-1 with LazyTensor? Thanks for any tips.

[Balandat]

I don't think there is a general way to do this. One could implement a reciprocal function on the LazyTensor base class and then go through the code to see where there are tensors for which this could be done cheaply / smartly.

I'm curious, what specifically is your application?