Given X,y data finding the y_predicted on the VAE manifold. USE case: for active learning we have scalarizer values and we want to interpolate it on vae latent space.

[utkarshp1161]

Hi,

Given: X, y

train VAE on X
Goal: Get y_predicted over the latent manifold of VAE

idea:
X_mean, X_std = VAE(X)
fit a function(gp?) : X_mean ---------> y
now get y_predicted over the latent manifold by passing it through the GP

Can we have a function in : https://github.com/ziatdinovmax/pyroVED/blob/main/pyroved/models/ivae.py ?
def manifold2d_predicted_label()

Please let me know if something in unclear

[ziatdinovmax]

@utkarshp1161 how would it be different from DKL-VAE?

[utkarshp1161]

I want to fit a regression on grid of coordinates in the z space(latent space). Now we get both manifold2d and manifold2d_predicted.
However fitting a regression on decoded grid also makes sense.

Idea is to be able to interpolate in vae latent space and get corresponding labels, without end to end training(like in DKL-VAE).

For now, a regression on vae latents will be great to have. What do you suggest?

[ziatdinovmax]

Hm... once you get the encoded vector, and assuming you have your y vector, you can just run a standard regression model in PyTorch, correct? It doesn't sound like this part needs any pyroVED modules.

[utkarshp1161]

Right. I found manifold2d() in pyroved very nice unlike other vae implementations out there. manifold2d() {already present} and manifold2d_predicted() {proposed} have the same z sampling and the plots obtained from them will be great to compare side by side so it will be great to have them together. Maybe a "predict_on_latents" flag in manifold2d() which if True takes the (X, y) and gives both the plots and if false just gives the usual plot.

[ziatdinovmax]

Can you please make a simple notebook demonstrating this? To avoid introducing new dependencies, I suggest using Pyro's GP module for the regression (https://pyro.ai/examples/gp.html). It doesn't have advanced options such as multi-fidelity and heteroskedasticity, but should be enough for the current goal.

[utkarshp1161]

Hi, I am not very familiar with pyro so gave chatgpt a shot.

Notebook: https://github.com/utkarshp1161/pyroVED/blob/main/examples/VAE_gp.ipynb

[utkarshp1161]

Hi Maxim, I have uploaded a notebook. Can you take a look? Best, Utkarsh

…

On Mon, Jan 8, 2024 at 12:40 PM Maxim Ziatdinov ***@***.***> wrote: Can you please make a simple notebook demonstrating this? To avoid introducing new dependencies, I suggest using Pyro's GP module for the regression (https://pyro.ai/examples/gp.html). It doesn't have advanced options such as multi-fidelity and heteroskedasticity, but should be enough for the current goal. — Reply to this email directly, view it on GitHub <#53 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AMR7ZTVZFUUO6PGU7OKFSP3YNQVQJAVCNFSM6AAAAABBQLGSZWVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DANJTGEYDG> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[ziatdinovmax]

Hi @utkarshp1161 - thanks for sharing the notebook. The way I understand this is that it can be helpful at the encoding stage for new X inputs for which there are no labels. Something like this

X_new -> trained_vae-encoder -> GPR -> predicted label.

One comment here is that default GPR is not well suited for classification problems. But the good news is that most physical problems deal with (quasi-)continuous variables.

[utkarshp1161]

Right for new X inputs, thus we can also get a mapping on latent space. I am finding this sort of visualisation handy for structure property relationship [eg: scalarizer in hAE]. Do you recommend a more modular way to do this[also any other functionality you think might be good]?

[ziatdinovmax]

Let's start by adding self.encode_and_predict method to iVAE that will run GPR on top of the current encoding.

[ziatdinovmax]

Closed via #55