[Q] What's the text model corresponding to the image tower datacomp_xl_s13b_b90k?

[NightMachinery]

[gabrielilharco]

Hey @NightMachinery, the config for the text tower can be found at https://huggingface.co/laion/CLIP-ViT-L-14-DataComp.XL-s13B-b90K/blob/main/open_clip_config.json#L10

[NightMachinery]

Thanks! But which weights does it actually load? There is no hub ID in the config. Can you give me a reference to where the weights come from? Who released those weights? OpenAI?

[gabrielilharco]

We also train the text tower from scratch

[leng-yue]

Hi, Is there a training command we can refer to?

[gabrielilharco]

There are more details at https://github.com/mlfoundations/datacomp