MultiMedEval 1.0.0

What's Changed

• MMLU dataset, custom logger and dataloader, edge case for deepspeed by @corentin-ryr
• Refactoring and improving output by @corentin-ryr
• Additon of Chestxray14 to Image Classification by @fitzlithepius
• fix path non-reference in mimic-iii by @corentin-ryr
• Implementation of InputBatcher data class by @fitzlithepius
• Add CT-RATE Report Generation Task by @fitzlithepius
• changes to ct-rate reportgen by @corentin-ryr
• Added CT-RATE Classification Task by @fitzlithepius
• Fix hf token in engine by @corentin-ryr
• New BatcherOutput class by @corentin-ryr
• Added Task Segmentation by @fitzlithepius
• Added Dataset REFUGE by @fitzlithepius
• Imrpovements to CT-RATE & image classification tests by @corentin-ryr
• CT-CLIP baseline by @corentin-ryr
• Include the Radgraph dependency to remove version incompatibility by @corentin-ryr
• added lipro baseline by @corentin-ryr

New Contributors

• @fitzlithepius made their first contribution

Full Changelog: v0.1.1...v1.0.0

MultiMedEval 0.1.1

New features:

• Dynamic datasets: use any dataset formatted correctly and apply the metrics of a task "family" (QA, VQA, Report Comparison, Image Classification, and NLI). This feature adds more flexibility to MultiMedEval.
• Added Diff-VQA [Paper] to the list of supported tasks.
• Updated RadCliQ to reflect more closely the results in the [Paper]

In addition to the new features, we added a suite of unit tests and corrected some bugs.

MultiMedEval 0.1

v0.1.0

Removed BnB from benchmarking code