Why does training loss not converge at the first stage?

[fffanli]

I cannot find the other curated datasets in StyleGallery except for the wikiart subset. And why does the loss fail to converge when I only train on the wikiart dataset?Here is my training command.
accelerate launch --num_processes 3 --multi_gpu --mixed_precision "fp16"
tutorial_train_styleshot_stage_1.py
--pretrained_model_name_or_path="runwayml/stable-diffusion-v1-5/"
--image_encoder_path="StyleShot/laion/CLIP-ViT-H-14-laion2B-s32B-b79K"
--image_json_file="StyleShot/wikiart_only.jsonl"
--image_root_path="StyleShot/data/wikiart"
--mixed_precision="fp16"
--resolution=512
--train_batch_size=8
--dataloader_num_workers=4
--learning_rate=1e-05
--weight_decay=0.001
--output_dir="StyleShot/results"

[Jeoyal]

Hi @fffanli , thank you for your interest in our work. What do you mean by “the loss fails to converge”?

[fffanli]

Hi @fffanli , thank you for your interest in our work. What do you mean by “the loss fails to converge”?

hello！Thanks for your reply.I have a problem that the loss value is always unstable and fluctuates significantly.

[Jeoyal]

It’s normal when you continue training a diffusion model. You’d better check the visual results to see if the model has converged.

[fffanli]

It’s normal when you continue training a diffusion model. You’d better check the visual results to see if the model has converged.

oh,How can I check?Judge the training effect by using the checkpoints after training?

[Jeoyal]

Yes

[fffanli]

YesThank you so much! Does you mean that such fluctuations of the loss value during the training process are normal? I have another question about datasets: according to the Dataset structure , how can I obtain the curated versions of JourneyDB and MultiGen-20M datasets?

[Jeoyal]

Download the full JourneyDB and MultiGen-20M datasets and index them by given jsonl file.

[fffanli]

Download the full JourneyDB and MultiGen-20M datasets and index them by given jsonl file.

OK.Thank you very much!