[Question] What is the lengthscale initialization function for kernels?

[mohammad-saber]

Hi, thank you for reading my question.

When I define a kernel like below:

kernel = gpytorch.kernels.RBFKernel()

I noticed that the initial kernel lengthscale is 0.6931. It seems that this value is fixed for all kernels.
I would like to ask how this value is created.

If I want to change initial kernel lengthscale, do you recommend to use lengthscale prior?

[gpleiss]

Run kernel.initialize(lengthscale=2.0) to change the initial lengthscale value to 2.0.

[gpleiss]

import random
random_lengthscale = random.random()  # uniformly distributed between 0 and 1
kernel.initialize(lengthscale=random_lengthscale)

[mohammad-saber]

Thank you again.
Actually I would like to use the same default method that GPyTorch uses to set initial legthscale. But, it seems that initial lengthscale in GPyTorch is always a fixed value (= 0.6931).
My purpose is training model for several epochs, initialize kernel lengthscale randomly, and continue training for more epochs. I would like to test if there is any improvement.
Thank you again.

[patel-zeel]

Hi @mohammad-saber,

Just to clarify, initial raw length scales are set to 0 everywhere but because of log(1+exp(x)) transformation for positivity, lengthscale modifies to 0.6931. As per my understanding, lengthscale_prior does not take a scaler as an argument but a prior distribution from gpytorch.priors (I found an example in this notebook under Priors section).

@gpleiss
other libraries such as GPy allow setting lengthscale from arguments so this confusion seems valid. Also, if users do not manually check the lengthscale believing they have reinitialized it with lengthscale_prior, they may arrive at wrong conclusions from their results. In my opinion, one of the following ways can avoid this confusion:

• Type hints to kernel arguments
• Raising an informative error if anything other than gpytorch.priors is passed to lengthscale_prior

I can happily raise a PR if one of the above ways seems reasonable to you.

[mohammad-saber]

Thank you very much for your help.

[gpleiss]

@patel-zeel both of those would be amazing! we'd happily accept a PR.