diff --git a/public_embedding_benchmarks/multi_modal_evals.md b/public_embedding_benchmarks/multi_modal_evals.md new file mode 100644 index 00000000..29beab4b --- /dev/null +++ b/public_embedding_benchmarks/multi_modal_evals.md @@ -0,0 +1,10 @@ +| Model | Dimensions| BEIR Average |Coco (2017) | Flickr1k | Design and Templates | E-Commerce | Graphs and Business Figures | Total Average | +| -------- | -------- | -------- | -------- | -------- | -------- | -------- | -------- | -------- | +| clip-vit-base-patch16 | 512 | 17.73% | 68.94%| 92.78% | 75.24% | 59.49% | 57.93% | 58.25% +| nomic-embed-vision-v1.5 | 768 | 47.87% | 65.24% | 87.18% | 65.34% | 64.84% | 36.48% | 58.15% +| jina-clip-v1 | 768 | 55.00% | 76.50% | 93.86% | 71.60% | 73.77% | 44.57% | 66.02% +| embed-english-v3.0 | 1024 | 55.9% |80.25% | 97.00% | 83.36% | 70.69% | 76.17% | 75.04% +| embed-multilingual-v3.0 | 1024 | 54.6% | 79.77% | 97.30% | 82.39% | 68.18% | 76.84% | 74.18% + + +Note: All Evaluations were calculated using Recall@10; Design and Templates, E-Commerce and Graphs and Business Figures are a composite of benchmarks gathered and annotated by Cohere \ No newline at end of file