Incorporating measurement error with spectral mixture kernel

[doccosmos]

What is the recommended method to add measurement error (prefereably heteroscedastic) to a spectral mixture model? I've tried:

specmix = gpytorch.kernels.SpectralMixtureKernel(num_mixtures=4)
error = gpytorch.kernels.WhiteNoiseKernel(train_yerr ** 2.)
self.covar_module = specmix + error

but this does not give a good fit and initialize_from_data does not then work.

[Balandat]

So the WhiteNoiseKernel has some issues, I wouldn't recommend using that right now. I'm working on replacing it by allowing you to implement likelihoods based on general noise models (e.g. themselves GPs). See #337 for this. This is currently blocked on a few issues with LazyTensors that need to be ironed out, I'm working on determining what we need to ship this.

[Balandat]

Disclaimer: I have not tested this with SpectralMixtureKernels, only with basic kernels so far.

[jacobrgardner]

If the issue is the fit not being good, the problem may also just be the SM kernel, which is extremely sensitive to hyperparameter settings. One thing that might be necessary (or at least was necessary in the matlab implementation) is a bunch of random restarts to find hyperparameters that actually work.

@doccosmos I could try to code up an example notebook doing this (probably just revamping our existing SM kernel notebook) in the next couple of days if that would be reasonable?

[AbhilashMathews]

I was just wondering if this had been resolved. In addition to incorporating measurement errors, are there any examples of the spectral mixture kernel applied in 2 dimensions and possibly for training and/or testing data on non-uniform grid points?

[jacobrgardner]

The original question has been resolved by using a FixedNoiseGaussianLikelihood, which is now on master.

The SM kernel can be applied to multiple dimensions but I can respond to the issue you already opened about this.

[doccosmos]

@jacobrgardner It would be useful to see how a notebook describing how to do this with a FixedNoiseGaussianLikelihood since:

specmix = gpytorch.kernels.SpectralMixtureKernel(num_mixtures=4)
error = gpytorch.likelihoods.FixedNoiseGaussianLikelihood(train_yerr ** 2.)
self.covar_module = specmix + error
self.covar_module.initialize_from_data(train_x, train_y)

Traceback (most recent call last):
File "/Users/mjg/anaconda3/lib/python3.6/site-packages/gpytorch/module.py", line 291, in getattr
return super().getattribute(name)
AttributeError: 'AdditiveKernel' object has no attribute 'initialize_from_data

[gpleiss]

specmix + error is the sum of a spectral mixture kernel and a likelihood. Those shouldn't be added together. Use the spectral mixture kernel as your kernel, and use the likelihood as your likelihood (in place of the standard FixedNoiseGaussianLikelihood.

[doccosmos]

So for a data set with (train_x, train_y, train_yerr):

likelihood = gpytorch.likelihoods.FixedNoiseGaussianLikelihood(noise = train_yerr)
model = gpytorch.kernels.SpectralMixtureKernel(train_x, train_y, likelihood)

[gpleiss]

gpytorch.kernels.SpectralMixtureKernel(train_x, train_y, likelihood) is not a model - it's a kernel.
I'd recommend following the spectral mixture kernel example notebook, and just replace the likelihood in there with gpytorch.likelihoods.FixedNoiseGaussianLikelihood(noise = train_yerr)

[doccosmos]

Thanks - that is what I was hoping the case but the documentation is not too clear on what noise is - is it the standard deviation or the variance or the inverse weights? — Matthew

…

On May 8, 2019, at 10:55 AM, Geoff Pleiss ***@***.***> wrote: gpytorch.kernels.SpectralMixtureKernel(train_x, train_y, likelihood) is not a model - it's a kernel. I'd recommend following the spectral mixture kernel example notebook, and just replace the likelihood in there with gpytorch.likelihoods.FixedNoiseGaussianLikelihood(noise = train_yerr) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#375 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAYXBCBMOUH6RPCAMK2QQZDPUMHYFANCNFSM4GD247DQ>.

[gpleiss]

It should be the variance.

[doccosmos]

OK - that works but when I run apply it to the test set (which has a different number of points): test_x = torch.linspace(train_x.min(), train_x.max(), 51) model.eval() likelihood.eval() with torch.no_grad(), gpytorch.settings.fast_pred_var(): observed_pred = likelihood(model(test_x)) /Users/mjg/anaconda3/lib/python3.6/site-packages/gpytorch/likelihoods/gaussian_likelihood.py:188: UserWarning:You have passed data through a FixedNoiseGaussianLikelihood that did not match the size of the fixed noise, *and* you did not specify noise. This is treated as a no-op. Note that the FixedNoiseGaussianLikelihood documentation applies it to a different sized data set and makes no changes to the model. — Matthew

…

On May 8, 2019, at 11:08 AM, Geoff Pleiss ***@***.***> wrote: It should be the variance. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#375 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAYXBCGT47I6QDF4Y46SCX3PUMJKHANCNFSM4GD247DQ>.

[gpleiss]

Is this only an issue with the spectral mixture kernel? What happens if you use an RBF?