Hyperparameter optimisation

[GMCobraz]

Dear caponetto and others,

Good day,
i am working on the hyperparameter optimisation.
I have some questions.
May I know, what is the g refers to in prior.py?
scatter_matrix = (data_matrix_cov / g).T

Thank you very much

[caponetto]

Hi @GMCobraz,
The way the prior is being created here comes from the original matlab code, where g scales empirical prior cov S.

[GMCobraz]

Hi @caponetto ,
Thanks for the reply.

Based on my understanding, cov_matrix = g * scatter_matrix, where g is 1/(N-1).
I am not understand why g for bhc and brt are different for the example, since N=18.

May I know why in prior.py, degrees_of_freedom = data.shape[1] + 1?

Is the scale factor can be understand as learning rate in this case?

Thank you very much.

[GMCobraz]

Hi @caponetto ,

May I know do you have an idea to implement the hyperparameter optimisation?
I am stuck halfway in implementing the brent method to optimize gamma for rose tree.
I try to implement this clustering method into my master thesis.
I will appreciate very much if you can give me some guidelines for the optimization

Thank you very much.

[caponetto]

Hi @GMCobraz,

Unfortunately, I don't have details about the behavior of the hyperparameters.
Maybe this is a good subject for exploring as part of your research?
If you do so, please share the results. :)

Also, notice that the example was taken from the original paper.
So I suppose that the hyperparameters are optimized for BHC.
For BRT, on the other hand, they are not optimized.
My goal was to use the same example for both strategies and provide an intuition.

Regarding the hyperparameter optimization itself, I suggest you take a look at Random Search [1].

[1] https://www.jmlr.org/papers/volume13/bergstra12a/bergstra12a.pdf