Results of Nemotron Embed VL 1B on Vidore V3.v2 tasks (page image + text)

[gabrielspmoreira]

This PR adds results of llama-nemotron-embed-vl-1b-v2 embedding model to MTEB Vidore V3.1 tasks. These 3.1 tasks provide both page image and text (OCR'ed) to the model.

Checklist

• My model has a model sheet, report, or similar
• My model has a reference implementation in mteb/models/model_implementations/nvidia_nemotron_vl_models.py
  • No, but there is an existing PR
• The results submitted are obtained using the reference implementation
• My model is available, either as a publicly accessible API or publicly on e.g., Huggingface
• I solemnly swear that for all results submitted I have not trained on the evaluation dataset including training splits. If I have, I have disclosed it clearly.

[github-actions]

Model Results Comparison

Reference models: intfloat/multilingual-e5-large, google/gemini-embedding-001
New models evaluated: nvidia/llama-nemotron-embed-vl-1b-v2
Tasks: Vidore3ComputerScienceRetrieval.v2, Vidore3EnergyRetrieval.v2, Vidore3FinanceEnRetrieval.v2, Vidore3FinanceFrRetrieval.v2, Vidore3HrRetrieval.v2, Vidore3IndustrialRetrieval.v2, Vidore3PharmaceuticalsRetrieval.v2, Vidore3PhysicsRetrieval.v2

Results for nvidia/llama-nemotron-embed-vl-1b-v2

task_name	nvidia/llama-nemotron-embed-vl-1b-v2	Max result	Model with max result	In Training Data
Vidore3ComputerScienceRetrieval.v2	0.6979			False
Vidore3EnergyRetrieval.v2	0.5867			False
Vidore3FinanceEnRetrieval.v2	0.5621			False
Vidore3FinanceFrRetrieval.v2	0.3731			False
Vidore3HrRetrieval.v2	0.5283			False
Vidore3IndustrialRetrieval.v2	0.3792			False
Vidore3PharmaceuticalsRetrieval.v2	0.6082			False
Vidore3PhysicsRetrieval.v2	0.4578			False
Average	0.5242		nan	-

---

---

Note: Content truncated due to GitHub API limits. See the full report in the workflow artifacts.

[KennethEnevoldsen]

results looks good - test fails model not being merged