[Doc] benchmarks detail doc

[Yunnglin]

No description provided.

[Copilot]

Pull request overview

This pull request introduces comprehensive documentation for benchmark datasets in the EvalScope evaluation framework. It adds detailed benchmark documentation files and updates the build system to support automated documentation generation.

Changes:

• Added 60+ new benchmark documentation files in docs/en/benchmarks/ covering diverse evaluation datasets (math, coding, multimodal, reasoning, etc.)
• Updated Makefile with new documentation generation targets and workflow automation
• Modified .pre-commit-config.yaml to exclude JSON files from end-of-file-fixer checks

Reviewed changes

Copilot reviewed 120 out of 637 changed files in this pull request and generated no comments.

File	Description
docs/en/benchmarks/*.md	Comprehensive documentation for 60+ benchmark datasets including properties, usage examples, and evaluation details
Makefile	Added documentation generation targets with parameters for translation, statistics, and selective updates
.pre-commit-config.yaml	Excluded JSON files from end-of-file-fixer to prevent formatting issues

[gemini-code-assist]

Summary of Changes

Hello @Yunnglin, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the project's documentation by integrating detailed information for a wide array of benchmarks. This includes task descriptions, key features, evaluation notes, and usage examples for each benchmark, making it easier for users to understand and utilize them. The underlying documentation generation process has also been modernized for better maintainability and flexibility.

Highlights

• Comprehensive Benchmark Documentation: Introduced extensive documentation for a wide array of benchmarks, detailing their purpose, task types, key features, evaluation metrics, and usage instructions. This significantly enhances the clarity and accessibility of information for each benchmark.
• Refactored Documentation Generation Workflow: The Makefile has been substantially updated to streamline the documentation generation process. New make targets (docs-update, docs-translate, docs-generate) leverage evalscope.cli.cli benchmark-info for more efficient and modular doc management.
• Pre-commit Configuration Update: Modified the pre-commit configuration to exclude JSON files from the end-of-file fixer hook, preventing unintended changes to JSON file formatting.
• Git Ignore Update: Added .qoder to the .gitignore file, ensuring that temporary or local development files are not committed to the repository.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request is a significant contribution, adding detailed documentation for a vast number of benchmarks and introducing new tooling to automate documentation generation, translation, and updates. The new Makefile targets and the underlying Python scripts are well-structured and will greatly improve the maintainability of the project's documentation.

My review focuses on two main areas for improvement:

1. Refactoring the Makefile for better readability and maintainability.
2. Improving the format of sample data in the generated documentation to avoid stringified JSON, which enhances clarity for users.

Overall, this is excellent work.

[gemini-code-assist]

The logic for docs-update, docs-update-stats, and docs-translate is quite repetitive. You can refactor this to make it more concise and maintainable by using variables for the arguments. This will reduce duplication and make it clearer how arguments like BENCHMARK and FORCE are handled.

BENCHMARK_ARG_UPDATE = $(if $(BENCHMARK),$(BENCHMARK),--all)
FORCE_ARG = $(if $(FORCE),--force,)

.PHONY: docs-update
docs-update:
	python -m evalscope.cli.cli benchmark-info $(BENCHMARK_ARG_UPDATE) --update

.PHONY: docs-update-stats
docs-update-stats:
	python -m evalscope.cli.cli benchmark-info $(BENCHMARK_ARG_UPDATE) --update --compute-stats

.PHONY: docs-translate
docs-translate:
	python -m evalscope.cli.cli benchmark-info $(BENCHMARK) --translate $(FORCE_ARG) --workers $(WORKERS)

[gemini-code-assist]

In the sample example JSON, the value for direct_answers is a string representation of a list, rather than a proper JSON array. This can be confusing for users trying to understand the data format. It would be more clear and correct to represent it as a JSON array.

This issue of stringifying complex data types appears in several other generated documentation files as well (e.g., target in drop.md, content in bfcl_v3.md). It would be great to fix the documentation generation script to output proper JSON structures for these fields.

Suggested change

	"direct_answers": "['cigarette', 'cigarette', 'cigarette', 'cigarette', 'cigarette', 'cigarette', 'cigarette', 'cigarette', 'cigarette', 'cigarette']",
	"direct_answers": [
	"cigarette",
	"cigarette",
	"cigarette",
	"cigarette",
	"cigarette",
	"cigarette",
	"cigarette",
	"cigarette",
	"cigarette",
	"cigarette"
	],