latent space

[smy123666]

Hello, I would like to know how you adapted the unet part of your SD to the latent space of your own trained vavae, because I have also been training my own vae recently, but I found that the VAE I trained does not apply well to the unet part of the original sd and has a very serious color deviation problem

[JingfengYao]

Hi,
I'm not sure if I fully understand what you mean. Every time we train a new VAE, we train a new Diffusion Model to test it. Directly replacing the VAE latent space is not feasible (at least in my opinion).

[smy123666]

Hi, I'm not sure if I fully understand what you mean. Every time we train a new VAE, we train a new Diffusion Model to test it. Directly replacing the VAE latent space is not feasible (at least in my opinion).

Thank you very much for your answer, I also plan to train my own unet on my VAE, so I would like to ask you, where did you get the prompts you need for UNET, and do you use fine-tuning methods such as controlnet, lora, etc. when training, in addition, I would like to know, after you train VAVAE, do you count the scaling factor of the encoder before training UNET?

[hjouqs]

Hi, I'm not sure if I fully understand what you mean. Every time we train a new VAE, we train a new Diffusion Model to test it. Directly replacing the VAE latent space is not feasible (at least in my opinion).

Thank you very much for your answer, I also plan to train my own unet on my VAE, so I would like to ask you, where did you get the prompts you need for UNET, and do you use fine-tuning methods such as controlnet, lora, etc. when training, in addition, I would like to know, after you train VAVAE, do you count the scaling factor of the encoder before training UNET?

I have the same problem, have you solved it?

[smy123666]

Hi, I'm not sure if I fully understand what you mean. Every time we train a new VAE, we train a new Diffusion Model to test it. Directly replacing the VAE latent space is not feasible (at least in my opinion).你好，我不太确定我是否完全理解你的意思。每次训练新的VAE时，我们都会训练新的扩散模型来测试它。直接替换VAE的潜在空间不可行（至少我个人认为是这样）。

Thank you very much for your answer, I also plan to train my own unet on my VAE, so I would like to ask you, where did you get the prompts you need for UNET, and do you use fine-tuning methods such as controlnet, lora, etc. when training, in addition, I would like to know, after you train VAVAE, do you count the scaling factor of the encoder before training UNET?非常感谢你的回答，我也计划在我的VAE上训练自己的unet，所以我想问一下，你从哪里获得UNET所需的提示？训练时你是否使用了像controlnet、lora等微调方法？另外，我想知道你训练VAVAE后， 你在训练UNET之前会计算编码器的缩放因子吗？

I have the same problem, have you solved it?我也有同样的问题，你解决了吗？

what the problem do you meet? i have not solve it