Adding pisa task

[HallerPatrick]

Hello!

this PR adds our benchmark PisaBench to the eval harness. It is a multi-lingual, multi-modal benchmark based on the PISA study. More information is provided in the task README.

Each instance contains a question and a complementary image and a set of multiple choice answers. Evaluation is done through substring matching, or by using gpt-4o-mini as a judged for each generated answer.

This task should be runnable with the model type hf-multimodal and vllm-vlm.

• Link to the HF orga containing the dataset and a leaderboard for current models
• Link to the paper

[CLAassistant]

All committers have signed the CLA.

[HallerPatrick]

Sorry for the delay. Didn't see that the CLA check didn't went through. I cannot interpret the error of the Tasks Modifiedcheck. Am I missing something?

[baberabb]

Sorry for the delay. Didn't see that the CLA check didn't went through. I cannot interpret the error of the Tasks Modifiedcheck. Am I missing something?

Could I ask you to remove the extension from lm_eval/tasks/pisa/_template.yaml. It currently expects all .yaml files to be proper configs, but this one is only used as an import in the other configs