I have a set of aligned embeddings, including image-text embeddings. You can consider them as the aligned results after CLIP processing. The recall rate R@1 for text-to-image retrieval is relatively high. However, after I started training, I noticed that the recall rate dropped significantly, and the loss barely decreased either. The loss remains consistently at 1.5561 (rl: 1.5561, vl: 0.0000). My training configuration is as follows:
--batch_size 1024
--steps 200000
--embed_dim 32
--hidden_dims 512 256 128
--codebook_size 256
--n_layers 3
--save_dir /root/workspace/work/niangao/projects/RQ-VAE-Recommender/out/rqvae_embedding_v2/
--save_every 10000
--lr 0.0001
I have a set of aligned embeddings, including image-text embeddings. You can consider them as the aligned results after CLIP processing. The recall rate R@1 for text-to-image retrieval is relatively high. However, after I started training, I noticed that the recall rate dropped significantly, and the loss barely decreased either. The loss remains consistently at 1.5561 (rl: 1.5561, vl: 0.0000). My training configuration is as follows:
--batch_size 1024
--steps 200000
--embed_dim 32
--hidden_dims 512 256 128
--codebook_size 256
--n_layers 3
--save_dir /root/workspace/work/niangao/projects/RQ-VAE-Recommender/out/rqvae_embedding_v2/
--save_every 10000
--lr 0.0001