Docs/sample reports #38
Conversation
…esults from git - Move oncology_translation_report.md to docs/applications/examples/ - Add reference to example report in zero_shot_evaluation.md - Add /evaluation_results to .gitignore - Fix examples/ gitignore rule to only ignore root-level examples folder
- Regenerate evaluation report in English using config.yaml - Update zero_shot_evaluation.md reference text to English - Replace Chinese example report with English version
Summary of ChangesHello @XiaoBoAI, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly improves the zero-shot evaluation feature by providing clear and comprehensive configuration examples, alongside detailed documentation for generating and understanding evaluation reports. It also includes a practical sample report to illustrate the output, making the evaluation process more accessible and transparent for users. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds extensive documentation and examples for the zero-shot evaluation report generation feature. This includes a new detailed configuration example, a minimal configuration example, a sample evaluation report, and updates to the main documentation to explain the new feature. The changes are well-structured and the new documentation is clear and comprehensive. The example configuration files are well-commented and follow good practices by using environment variables for API keys. I have a couple of minor suggestions for improving the formatting of the new Markdown files to ensure consistency and maintainability. Overall, this is a great addition to the project's documentation.
| This evaluation assessed the performance of three Qwen-series AI models—qwen-plus, qwen-turbo, and qwen3-32b—in translating oncology-related Chinese texts into English. The task required precise rendering of specialized terminology, including disease names, therapeutic regimens, drug nomenclature, and clinical metrics, while preserving the scientific rigor expected in medical literature. The scenario reflects real-world needs of researchers and clinicians preparing manuscripts or case reports for international publication. | ||
|
|
||
| The evaluation employed pairwise comparisons across 42 trials derived from 7 representative test queries, covering diverse oncology subdomains. Human evaluators judged translation accuracy, terminological fidelity, and stylistic appropriateness. Results showed qwen-plus significantly outperformed its counterparts, achieving a win rate of 64.3%. Both qwen-turbo and qwen3-32b tied at 42.9%, indicating comparable but notably lower performance. | ||
|
|
||
| qwen-plus demonstrated superior handling of complex medical concepts and consistent adherence to standard oncology nomenclature, making it the most reliable choice for high-stakes academic and clinical translation tasks in this domain. These findings support its adoption for professional medical translation workflows requiring precision and credibility. |
There was a problem hiding this comment.
There's inconsistent use of trailing double spaces for line breaks throughout this document. For example, lines 5 and 7 end with two spaces, while line 9 does not.
In Markdown, trailing double spaces create a hard line break (<br>). For creating distinct paragraphs, the standard and more maintainable approach is to use a blank line between them. The current mix can lead to inconsistent rendering and makes the source harder to read and edit.
It's recommended to remove all such trailing spaces and rely solely on blank lines for paragraph separation for consistency.
OpenJudge Version
[The version of OpenJudge you are working on, e.g.
import openjudge; print(openjudge.__version__)]Description
[Please describe the background, purpose, changes made, and how to test this PR]
Checklist
Please check the following items before code is ready to be reviewed.
pre-commit run --all-filescommand