Covariance Matrix from ExactGP is not Positive Definite

[VhaoYan]

Hi, I find the covariance matrix I got from the ExactGP is not Positive Definite, therefore if I want to create a new multivariance dist from the resulting (mean, cov) is not possible.

May I ask why the return value of ExactGP is from a MultivarianceDistribution() but if I check the covariance matrix from that distribution, it is not Positive Definite?

model = ExactGPModel(train_x, train_y, likelihood)
test = likelihood(model(test_x))
test_covariance = test.covariance_matrix <- not positive definite.
(the train_x and test_x are just randomly sampled from a uniform distribution)

[gpleiss]

This is expected due to numerical instabilities. All of the internals of GPyTorch add a small constant diagonal to covariance matrices as needed in order to make the covariance matrix positive definite.

Note that all of the covariance matrices are PD in exact arithmetic, but not necessarily PD in floating point precision due to round-off errors.