Multi-fidelity GP with general noise for all data and different specific noises for each group

[Mehdishishehbor]

Hi,
I have a stack of training data that are coming from different sources with different noise values. Is there a way to define additional noise parameters for each group separately and having one noise parameter for the entire data sets? For example, let's say I have 4 data resources with different noise level. I want 4 noise estimates for each level and 1 for all the data (5 in total). Does any of the likelihoods in Gpytorch handle this or can you suggest a likelihood that I can extent to solve this problem?

I appreciate your help and support.

Best,

Mehdi

[wjmaddox]

Your best bet is probably to use a FixedNoiseGaussianLikelihood and to set the noise estimates for each portion of your data. It also has the learn_additional_noise flag which can help you learn a global set of noise for all of the data.

[Mehdishishehbor]

Thanks for your help and suggestion. If I understand correctly, fixednoisegaussianlikelihood only calculates one general noise and the given noises for each portion of the data are known. Is there anyway to make those given noises unknown and find them through optimization?

[wjmaddox]

Not without writing your own gpytorch.likelihoods.Noise specialized class. In general, this might be somewhat ill-defined and better suited to include the four groups as an identifier in your GP itself, rather than tossing it in the noise setting.

I can think of two semi-hacky ways to go about it: 1) you could have 4 separate GPs but share the mean and kernel between them (possibly something like alternating learning of the hypers b/w the models) or 2) you could sub-class FixedNoiseGaussianLikelihood to allow you to make and expand the noise tensor as you wish. The second might actually get you exactly what you want but might be tricky to setup.

[Mehdishishehbor]

Thank you for your kind response. It was very helpful.