refactor(multimodal): change score range from [0,1] to 1-5 scale

[XiaoBoAI]

• Update ImageCoherenceGrader to use 1-5 scoring
• Update ImageHelpfulnessGrader to use 1-5 scoring
• Update TextToImageGrader to use 1-5 scoring
• Update documentation and tests accordingly

OpenJudge Version

[The version of OpenJudge you are working on, e.g. import openjudge; print(openjudge.__version__)]

Description

[Please describe the background, purpose, changes made, and how to test this PR]

Checklist

Please check the following items before code is ready to be reviewed.

• Code has been formatted with pre-commit run --all-files command
• All tests are passing
• Docstrings are in Google style
• Related documentation has been updated (e.g. links, examples, etc.)
• Code is ready for review

[gemini-code-assist]

Summary of Changes

Hello @XiaoBoAI, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a significant refactoring of the scoring mechanism across several multimodal graders. The primary goal is to standardize the output scores to a more intuitive and widely understood 1-5 scale, moving away from the previous 0-1 or 0-10 ranges. This change simplifies score interpretation and ensures consistency across different evaluation metrics within the multimodal grading system.

Highlights

• Unified Scoring Scale: The scoring range for ImageCoherenceGrader, ImageHelpfulnessGrader, and TextToImageGrader has been refactored from a [0,1] or [0,10] scale to a consistent 1-5 scale.
• Documentation Updates: All relevant documentation, including overview tables, detailed grader descriptions, scoring criteria, and example code snippets, has been updated to reflect the new 1-5 scoring system.
• Internal Logic Adjustments: The internal prompt instructions for LLM-based graders, default neutral scores for missing fields, and the final score calculation logic (especially for TextToImageGrader) have been modified to align with the 1-5 scale, removing previous normalization steps.
• Test Suite Alignment: Unit and batch evaluation tests for the affected multimodal graders have been updated to assert scores within the new 1-5 range and use appropriate mock values.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request successfully refactors the multimodal graders (ImageCoherenceGrader, ImageHelpfulnessGrader, TextToImageGrader) to use a 1-5 scoring scale instead of the previous [0,1] normalized scale. The changes are consistently applied across the grader implementations, documentation, and tests. The core logic for score calculation has been updated, and prompts and docstrings now reflect the new 1-5 scale.

While the changes within the diff are correct, I've identified a few related areas that might need attention to fully complete this refactoring:

• In ImageCoherenceGrader and ImageHelpfulnessGrader, error cases (like no images found) result in a score of 0.0, which is outside the new 1-5 range. This should likely be 1.0 for consistency with TextToImageGrader.
• In TextToImageGrader, the aevaluate method returns a score of 0.0 for invalid inputs, which is also out of range. Additionally, the generated reason string for this grader still refers to scores out of /10.

Since these issues are on lines not modified in this pull request, I haven't added specific comments for them. However, addressing them would make this refactoring more complete and robust. I've added one specific comment regarding a documentation change.

[gemini-code-assist]

The ImageEditingGrader has been removed from this overview table. This seems to be an undocumented change as the PR description doesn't mention it. If this removal is intentional, please consider updating the PR description. If it's accidental, please add it back.