Commit b703548
RM Gallery V2
Link: https://code.alibaba-inc.com/OpenRepo/RM-Gallery/codereview/24203053
* [New] V2
* [update] documents
* [update] data sample schema
* Initial commit
* [update readme and docs]
* Merge pull request #1 from modelscope/doc_dev
[update] readme and docs
* [update] update readme (#2)
* [update] update readme
* [update] update readme
* [fix] reward register (#3)
fix import
* [update] update git remote url (#4)
* [update] update readme
* [update] update readme
* [update] update git remote url
* fix: resolve import errors in dataset module
* Merge pull request #6 from modelscope/pairwise
fix: resolve import errors in dataset module
* [feat] add async evaluation support for reward modules with semaphore-based concurrency control (#8)
* [feat] add async evaluation support for reward modules with semaphore-based concurrency control
* [fix] Fixed the bug in reward post-processing in asynchronous reward calculation
* [new] add bradley-terry and sft scripts (#13)
* [new] add bradley-terry and sft scripts
* [new] add bradley-terry and sft scripts
* [fix] fix dependencies (#12)
* [fix] fix docs (#11)
* [update] bt train scripts (#14)
* [new] add bradley-terry and sft scripts
* [new] add bradley-terry and sft scripts
* [update] bt train scripts
* [update] bt train scripts
* [delete] old bt scripts
* [update] sft_rm.md
* [delete] duplicate sft folders (#16)
* update llm bench
* add
* feat: Add Ties subset support and parallel processing to RewardBench2
* feat: Add Ties subset support and parallel processing to RewardBench2
* feat: Add Ties subset support and parallel processing to RewardBench2
* feat: Add Ties subset support and parallel processing to RewardBench2
* feat: Add Principles for rewardbench2
* [fixbug] pointwise dataset (#18)
* [delete] duplicate sft folders
* [fixbug] pointwise dataset
* refactor: cleanup evaluation modules and update documentation
- Remove deprecated evaluation modules (conflict_detector, judgebench, rmb, rmbench)
- Update rewardbench2 evaluation module and documentation
- Clean up template modules
- Update load.ipynb tutorial
- Fix linting issues in rewardbench2.py
* Merge pull request #19 from modelscope/llm_bench
Llm bench
* feat: upgrade documentation theme to mkdocs-shadcn
- Migrate from material theme to mkdocs-shadcn for modern UI
- Enhance homepage with gradient logo design and Inter font
- Standardize badge styles and layout structure
- Add GitHub Actions workflow for automated deployment
- Improve visual consistency and user experience
- Configure markdown extensions for rich content support
* feat: implement coy theme for code highlighting
- Add Prism.js coy theme for modern code block styling
- Configure enhanced syntax highlighting with line numbers
- Create custom CSS enhancements for better visual appeal
- Support multiple programming languages with autoloader
- Add responsive design for mobile devices
- Implement hover effects and improved readability
* feat: enhance code block styling with copy functionality
- Add code copy button feature for better UX
- Implement One Dark Pro syntax highlighting theme
- Include JetBrains Mono font for better code readability
- Add custom CSS for enhanced code block appearance
- Configure pymdownx.highlight with line numbers and anchors
- Add responsive design for code blocks on mobile devices
* feat: add interactive code copy button functionality
- Implement custom JavaScript for code block copy functionality
- Add hover-triggered copy button with smooth animations
- Include visual feedback with check icon on successful copy
- Style copy button with modern design and transitions
- Support both custom and theme-native copy button styles
- Ensure cross-browser clipboard API compatibility
* feat: optimize table rendering with text wrapping
- Add comprehensive table styling with proper text wrapping
- Enable word-break and overflow-wrap for all table cells
- Implement responsive table design for mobile devices
- Add hover effects and striped rows for better readability
- Include gradient header background for visual appeal
- Configure tables markdown extension for proper rendering
- Add smooth scrolling for wide tables on small screens
* fix: restore complete documentation content from main branch
- Recover all original documentation sections and content
- Preserve installation guide, walkthrough, and examples
- Maintain documentation table and citation information
- Keep all code examples and detailed explanations
- Apply modern styling only to header section without content loss
* docs: Add new documentation sections and update mkdocs configuration
- Add rm_library.md and rubric_library.md in library section
- Add navigation.md for improved site navigation
- Add boosting_strategy.md in using_rm section
- Add reference section with .gitkeep
- Update mkdocs.yml configuration
* feat: create interactive RM and Rubric libraries
- Transform static markdown pages into dynamic interactive libraries
- Add search and filter functionality similar to ReMe project design
- RM Library: categorized display of reward models with detailed info
- Rubric Library: comprehensive evaluation rubrics with principles
- Modern responsive UI with modal details and real-time stats
- Consistent with navigation.md planning structure
* [new] auto-rubric
* [rename] rubric
* [new] auto-rubric (#21)
* [new] auto-rubric
* [rename] rubric
* feat: add LLM Judge evaluation module and RL training examples
- Add llm_judge module with pointwise/pairwise/listwise evaluators
- Add alignment reward functions for LLM judge
- Add RL training examples with alignment reward integration
- Add reward manager and alignment RL dataset
- Add GRPO training script and documentation
* refactor: improve RL training dataset and reward function
- Add base dataset class for RL training
- Refactor alignment dataset with DataKeys configuration
- Improve code formatting and structure
- Update reward function documentation
* update
* fix: improve base dataset import with fallback mechanism
- Add robust import fallback for base_dataset module
- Update README and reward manager
- Improve error handling for module imports
* fix
* Update RL training and LLM judge evaluation modules
* [update] rubric src
* Merge branch 'main' into autorubric_gt
* Add evaluation tools and documentation
- Add conflict_detector evaluation tool
- Add judgebench, rmb, rmbench evaluation modules
- Add documentation for evaluation methods
- Add llm_judge reward modules
- Update rewardbench2 implementation
- Add RL training examples
- Fix linting issues (unused imports, f-string formatting)
* Merge llm_bench into feature/upgrade-docs-theme
* Merge origin/boyin_dgr into feature/upgrade-docs-theme
- Add LLM judge framework with adapters, evaluators, and templates
- Add reward manager and RL training examples
- Add base dataset for RL training
- Resolve conflict in alignment_rl_dataset.py
* docs: convert tutorial notebooks to markdown and update documentation
* feat: improve rubric library UI - optimize chip display and layout
* [update] rubric_library
* feat: convert all Jupyter notebooks to Markdown format
- Convert 7 .ipynb files to .md format for better version control
- Update mkdocs.yml to reference .md files instead of .ipynb
- Optimize RM Library card styles (simplified tags, improved layout)
- Update Building RM navigation structure
Files converted:
- tutorial/data: annotation, load, pipeline, process
- tutorial/rm_application: best_of_n, data_refinement, post_training
Benefits:
- Faster build times (no Jupyter conversion needed)
- Better git diffs and version control
- Easier editing and maintenance
- Simplified dependencies
* fix(judgebench): fix evaluation results not being stored in batch processing
- Override _async_parallel method in JudgeBenchReward to use BaseListWiseReward implementation
- Fixes issue where BaseLLMReward._async_parallel was storing results in wrong location due to MRO
- Results now correctly stored in sample.input[-1].additional_kwargs for compute_accuracy
- Tested with qwen2.5-32b-instruct via DashScope API, accuracy calculation now works correctly
* docs(judgebench): add custom API endpoint configuration example
- Add example showing how to configure base_url for custom API endpoints
- Demonstrates usage with Alibaba Cloud DashScope API
- Helps users who need to use OpenAI-compatible third-party APIs
* feat: upgrade docs theme and add conflict detector improvements
- Update mkdocs.yml with new theme configuration
- Enhance documentation pages (index, rm_library, rubric_library, boosting_strategy)
- Add search-fix.js for improved search functionality
- Improve conflict_detector.py with new features
- Add template.py for evaluation
- Add comprehensive test files for conflict detector
* Merge branch 'main' into feature/upgrade-docs-theme
- Resolved conflicts in docs/index.md, mkdocs.yml, and autorubric.md
- Updated all .ipynb references to .md files
- Removed .ipynb files that were converted to .md
- Integrated rubric-related updates from main branch
* refactor: unify terminology from 'principle' to 'rubric' across codebase
- Updated code files: rmb.py, rmbench.py
- Changed PrincipleListWiseTemplate to RubricListWiseTemplate
- Updated class inheritance and type annotations
- Updated documentation files:
- Renamed autoprinciple.md to autorubric.md
- Updated overview.md: AutoPrinciple → AutoRubric, generator variables
- Updated custom_reward.md: BasePrincipleReward → BaseRubricReward
- Updated evaluation/overview.md, best_of_n.md, post_training.md, boosting_strategy.md
- Updated rm_library.md: CSS, JS, HTML elements, and RM configurations
- All terminology now consistently uses 'rubric' instead of 'principle'
- This change improves clarity and consistency in the reward modeling framework
* Update conflict detector and add test files
* docs: update documentation and add examples
- Update main README and documentation index
- Add FAQ and quickstart guides
- Add tutorial documentation and end-to-end guide
- Add example notebooks (quickstart, custom-rm, evaluation)
- Add README files for rm modules
- Remove outdated POINTWISE_CONFLICT_ANALYSIS.md
* docs: update documentation files and add sitemap
- Update FAQ, quickstart, and tutorial documentation
- Update docs index and mkdocs configuration
- Add sitemap.txt for documentation
* docs: restructure navigation following Diataxis framework
- Reorganize navigation from nested 3-level to flat 2-level structure for better compatibility with shadcn theme
- Replace 'How-to Guides' with topic-based sections: Building RM, Training RM, Evaluating RM, Data Processing, RM Applications
- Improve navigation clarity and user experience
- Keep Tutorials section focused on end-to-end learning
- All documentation files remain accessible with clearer categorization
* docs: improve documentation and jupyter notebooks
- Add code zoom functionality for better code viewing
- Enhance CSS styles for better readability
- Add new jupyter-simple.css for notebook styling
- Update README.md
- Update example notebooks (custom-rm, evaluation, quickstart)
- Add CHANGELOG.md
* docs: add evaluation frameworks comparison analysis and LLM judge research survey
* docs: 优化文档结构和内容
主要改进:
- 重构首页:精简内容从 489 行减至 236 行(52%↓)
- 优化导航:将 Reference 部分前置,删除不存在的 API Documentation 和 Changelog
- 清理 Jupyter Notebook 引用:删除所有 .ipynb 文件引用,修复 16+ 处错误链接
- 简化 Learning Paths:移除冗余子项描述,使路径更清晰
- 修复 Installation tabs:统一使用 pymdownx.tabbed 语法,移除扩展冲突
- 精简 Tutorial README:从 242 行减至 175 行(28%↓)
- 统一文档格式:将 'notebook' 改为 'guide',保持一致性
影响的文件:
- 核心文档:index.md, quickstart.md, mkdocs.yml
- 教程文档:tutorial/README.md 及多个子教程
- 配置文件:sitemap.txt
这些改进让文档更加简洁、准确、易于导航。
* docs: Add navigation scroll fix and update rubric library with dataset link
* Merge pull request #23 from modelscope/docs_diataxis
Docs diataxis
* feat: 添加 GitHub Pages 自动部署配置,移除 Jupyter Notebook 引用
- 添加 GitHub Actions 工作流自动部署文档到 GitHub Pages
- 创建 docs/requirements.txt 管理文档依赖
- 从 mkdocs.yml 移除 mkdocs-jupyter 插件
- 删除 docs/examples 符号链接,避免包含 .ipynb 文件
- 添加部署总结文档
* Revert "feat: 添加 GitHub Pages 自动部署配置,移除 Jupyter Notebook 引用"
This reverts commit 16603fa.
* feat: add GitHub Pages deployment workflow
- Add GitHub Actions workflow for automatic deployment
- Create docs/requirements.txt for documentation dependencies
- Remove mkdocs-jupyter plugin from mkdocs.yml
- Remove docs/examples symlink to exclude .ipynb files from docs
- Use latest action versions (v4, v5) to avoid deprecation warnings
* Revert "feat: add GitHub Pages deployment workflow"
This reverts commit b0bd2de.
* chore: configure documentation deployment and update mkdocs settings
- Add GitHub Actions workflow for automated docs deployment
- Add docs/requirements.txt for documentation dependencies
- Remove mkdocs-jupyter plugin from mkdocs.yml
- Update sft_rm.md documentation
- Remove docs/examples symlink
- Add .env to .gitignore
* Merge pull request #22 from modelscope/autorubric_gt
[update] autorubric src
* fix: center modal dialog in RM Library
- Fix modal positioning to display in center of viewport
- Use fixed positioning with top/left 50% and transform translate
- Remove ineffective flex/inset properties that were preventing centering
- Set margin to 0 to avoid offset interference
* chore: remove obsolete files
Remove unused research notes, changelog, and test files:
- 2025_LLM_as_Judge_Agent_Research_Survey.md
- CHANGELOG.md
- CONFLICT_DETECTOR_SUMMARY.md
- test_10_samples.py
- test_conflict_detector_comprehensive.py
* [update] mapper
* [update] fix bugs
* update
* Remove agentscope submodule
* update agentscope
* update schema
* add test
* update readme
* add optimizer
* fix bug
* fix bugs
* update template
* update grader
* fix bugs
* fix bugs
* update voting
* remove old files1 parent af78a01 commit b703548
File tree
179 files changed
+2746
-26631
lines changed- .github/workflows
- docs
- images
- building_rm
- rm_server
- tutorial
- building_rm
- data
- rm_application
- rm_serving
- training_rm
- examples
- data
- rm_application
- train
- pairwise
- pointwise
- external
- rm_gallery
- core
- data
- annotation
- load
- process
- ops
- filter
- model
- optimizer
- reward
- principle
- train
- utils
- gallery
- data
- annotation
- load
- rm
- alignment
- harmlessness
- helpfulness
- honesty
- code
- prime_code
- format
- math
- tests
- data
- rm
- test
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
179 files changed
+2746
-26631
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
| 11 | + | |
11 | 12 | | |
12 | 13 | | |
13 | 14 | | |
14 | 15 | | |
15 | 16 | | |
16 | 17 | | |
17 | 18 | | |
18 | | - | |
| 19 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
0 commit comments