Add dapo reward #114

hiyuchang · 2025-07-10T06:00:17Z

Description

As the title says.

Note:

set sync_interval=16 to achieve: For training, the mini-batch size is set to 512, i.e., 16 gradient updates for each rollout.
The DAPO-Math dataset is pre-transformed into {'prompt': xxx, 'ground_truth': xxx}

Checklist

Please check the following items before code is ready to be reviewed.

Code has passed all tests
Docstrings have been added/updated in Google Style
Documentation has been updated
Code is ready for review

hiyuchang · 2025-07-10T07:21:12Z

/run-unittest

github-actions · 2025-07-10T07:21:52Z

Summary

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Other ❓	Flaky 🍂	Duration ⏱️
0	0	0	0	0	0	4ms

Tests

Test Name	Status	Flaky	Duration

Github Test Reporter by CTRF 💚

chenyushuo · 2025-07-10T07:29:11Z

/run-unittest

github-actions · 2025-07-10T07:29:51Z

Summary

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Other ❓	Flaky 🍂	Duration ⏱️
0	0	0	0	0	0	4ms

Tests

Test Name	Status	Flaky	Duration

Github Test Reporter by CTRF 💚

hiyuchang · 2025-07-10T09:20:59Z

/run-unittest

github-actions · 2025-07-10T09:41:44Z

Summary

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Other ❓	Flaky 🍂	Duration ⏱️
40	40	0	0	0	0	1.2s

Tests

Test Name	Status	Duration
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_dpo_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_mix_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_opmd_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_ppo_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_sft_policy_loss	✅	1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_buffer	✅	3ms
tests/buffer/file_test.py::TestFileBuffer::test_file_reader	✅	1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_writer	✅	1ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer	✅	5ms
tests/buffer/sql_test.py::TestSQLBuffer::test_create_sql_buffer	✅	3ms
tests/common/config_test.py::TestConfig::test_all_examples_are_valid	✅	1ms
tests/common/config_test.py::TestConfig::test_load_default_config	✅	6ms
tests/common/experience_test.py::TestExperienceConversion::test_batch_conversion	✅	1ms
tests/common/experience_test.py::TestExperienceConversion::test_experience_model_experience_conversion	✅	1ms
tests/common/vllm_test.py::TestModelWrapperSyncV0::test_generate	✅	42ms
tests/common/vllm_test.py::TestModelWrapperAsyncV0::test_generate	✅	40ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV0::test_generate	✅	50ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV1::test_generate	✅	50ms
tests/common/vllm_test.py::TestModelWrapperAsyncV1::test_generate	✅	38ms
tests/common/vllm_test.py::TestAPIServer::test_api	✅	24ms
tests/common/vllm_test.py::TestTokenizer::test_assistant_token_mask	✅	1ms
tests/explorer/explorer_test.py::BaseExplorerCase::test_explorer	✅	1ms
tests/explorer/explorer_test.py::TestExplorerCountdownEval::test_explorer	✅	123ms
tests/explorer/explorer_test.py::TestExplorerCountdownNoEval::test_explorer	✅	96ms
tests/explorer/runner_pool_test.py::RunnerPoolTest::test_runner_pool	✅	22ms
tests/explorer/runner_pool_test.py::RunnerPoolTest::test_runner_pool_with_auxiliary_models	✅	4ms
tests/explorer/workflow_test.py::WorkflowTest::test_gsm8k_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_boxed_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_complex_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_fraction_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_workflow_resettable	✅	1ms
tests/trainer/trainer_test.py::BaseTrainerCase::test_trainer	✅	1ms
tests/trainer/trainer_test.py::TestTrainerCountdown::test_trainer	✅	271ms
tests/trainer/trainer_test.py::TestStepAheadAsyncRL::test_trainer	✅	103ms
tests/trainer/trainer_test.py::TestTrainerGSM8K::test_trainer	✅	60ms
tests/trainer/trainer_test.py::TestTrainerSFTWarmupGSM8K::test_trainer	✅	92ms
tests/trainer/trainer_test.py::TestTrainerDPO::test_trainer	✅	42ms
tests/trainer/trainer_test.py::TestFullyAsyncMode::test_fully_async_mode	✅	110ms
tests/utils/plugin_test.py::TestPluginLoader::test_load_plugins	✅	4ms

Github Test Reporter by CTRF 💚

trinity/common/workflows/dapo_workflow.py

hiyuchang · 2025-07-14T11:29:38Z

/run-unittest

github-actions · 2025-07-14T11:52:14Z

Summary

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Other ❓	Flaky 🍂	Duration ⏱️
51	50	1	0	0	0	1.3s

Failed Tests

Failed Tests ❌	Fail Message
❌ tests/common/config_test.py::TestConfig::test_all_examples_are_valid	The test failed in the call phase due to an exception

Tests

Test Name	Status	Duration
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_dpo_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_mix_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_opmd_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_ppo_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_sft_policy_loss	✅	1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_buffer	✅	2ms
tests/buffer/file_test.py::TestFileBuffer::test_file_reader	✅	1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_writer	✅	2ms
tests/buffer/queue_test.py::TestQueueBuffer::test_priority_queue_buffer_reuse	✅	7ms
tests/buffer/queue_test.py::TestQueueBuffer::test_priority_queue_capacity	✅	3ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer_0_queue	✅	4ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer_1_priority_queue	✅	4ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer_capacity	✅	4ms
tests/buffer/sql_test.py::TestSQLBuffer::test_create_sql_buffer	✅	5ms
tests/common/config_test.py::TestConfig::test_all_examples_are_valid	❌	1ms
tests/common/config_test.py::TestConfig::test_load_default_config	✅	4ms
tests/common/experience_test.py::TestExperienceConversion::test_batch_conversion	✅	1ms
tests/common/experience_test.py::TestExperienceConversion::test_experience_model_experience_conversion	✅	1ms
tests/common/vllm_test.py::TestModelWrapperSyncV0::test_generate	✅	44ms
tests/common/vllm_test.py::TestModelWrapperAsyncV0::test_generate	✅	43ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV0::test_generate	✅	54ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV1::test_generate	✅	56ms
tests/common/vllm_test.py::TestModelWrapperAsyncV1::test_generate	✅	42ms
tests/common/vllm_test.py::TestAPIServer::test_api	✅	28ms
tests/common/vllm_test.py::TestTokenizer::test_assistant_token_mask	✅	1ms
tests/explorer/explorer_test.py::BaseExplorerCase::test_explorer	✅	1ms
tests/explorer/explorer_test.py::TestExplorerCountdownEval::test_explorer	✅	92ms
tests/explorer/explorer_test.py::TestExplorerCountdownNoEval::test_explorer	✅	84ms
tests/explorer/scheduler_test.py::SchedulerTest::test_concurrent_operations	✅	5ms
tests/explorer/scheduler_test.py::SchedulerTest::test_get_results	✅	21ms
tests/explorer/scheduler_test.py::SchedulerTest::test_scheduler_all_methods	✅	15ms
tests/explorer/scheduler_test.py::SchedulerTest::test_scheduler_restart_after_stop	✅	9ms
tests/explorer/scheduler_test.py::SchedulerTest::test_split_tasks	✅	8ms
tests/explorer/scheduler_test.py::SchedulerTest::test_wait_all	✅	8ms
tests/explorer/scheduler_test.py::SchedulerTest::test_wait_all_timeout_with_multi_batch	✅	14ms
tests/explorer/workflow_test.py::WorkflowTest::test_gsm8k_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_boxed_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_complex_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_fraction_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_rm_gallery_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_workflow_resettable	✅	1ms
tests/trainer/trainer_test.py::BaseTrainerCase::test_trainer	✅	1ms
tests/trainer/trainer_test.py::TestTrainerCountdown::test_trainer	✅	248ms
tests/trainer/trainer_test.py::TestStepAheadAsyncRL::test_trainer	✅	89ms
tests/trainer/trainer_test.py::TestTrainerGSM8K::test_trainer	✅	63ms
tests/trainer/trainer_test.py::TestTrainerSFTWarmupGSM8K::test_trainer	✅	88ms
tests/trainer/trainer_test.py::TestTrainerDPO::test_trainer	✅	44ms
tests/trainer/trainer_test.py::TestFullyAsyncMode::test_fully_async_mode_0_queue	✅	103ms
tests/trainer/trainer_test.py::TestFullyAsyncMode::test_fully_async_mode_1_priority_queue	✅	98ms
tests/utils/plugin_test.py::TestPluginLoader::test_load_plugins	✅	5ms

Github Test Reporter by CTRF 💚

hiyuchang · 2025-07-14T11:54:22Z

/run-unittest

github-actions · 2025-07-14T12:17:01Z

Summary

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Other ❓	Flaky 🍂	Duration ⏱️
51	51	0	0	0	0	1.3s

Tests

Test Name	Status	Duration
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_dpo_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_mix_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_opmd_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_ppo_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_sft_policy_loss	✅	1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_buffer	✅	3ms
tests/buffer/file_test.py::TestFileBuffer::test_file_reader	✅	1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_writer	✅	2ms
tests/buffer/queue_test.py::TestQueueBuffer::test_priority_queue_buffer_reuse	✅	6ms
tests/buffer/queue_test.py::TestQueueBuffer::test_priority_queue_capacity	✅	2ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer_0_queue	✅	5ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer_1_priority_queue	✅	5ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer_capacity	✅	4ms
tests/buffer/sql_test.py::TestSQLBuffer::test_create_sql_buffer	✅	5ms
tests/common/config_test.py::TestConfig::test_all_examples_are_valid	✅	1ms
tests/common/config_test.py::TestConfig::test_load_default_config	✅	4ms
tests/common/experience_test.py::TestExperienceConversion::test_batch_conversion	✅	1ms
tests/common/experience_test.py::TestExperienceConversion::test_experience_model_experience_conversion	✅	1ms
tests/common/vllm_test.py::TestModelWrapperSyncV0::test_generate	✅	44ms
tests/common/vllm_test.py::TestModelWrapperAsyncV0::test_generate	✅	42ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV0::test_generate	✅	53ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV1::test_generate	✅	55ms
tests/common/vllm_test.py::TestModelWrapperAsyncV1::test_generate	✅	42ms
tests/common/vllm_test.py::TestAPIServer::test_api	✅	28ms
tests/common/vllm_test.py::TestTokenizer::test_assistant_token_mask	✅	1ms
tests/explorer/explorer_test.py::BaseExplorerCase::test_explorer	✅	1ms
tests/explorer/explorer_test.py::TestExplorerCountdownEval::test_explorer	✅	80ms
tests/explorer/explorer_test.py::TestExplorerCountdownNoEval::test_explorer	✅	90ms
tests/explorer/scheduler_test.py::SchedulerTest::test_concurrent_operations	✅	5ms
tests/explorer/scheduler_test.py::SchedulerTest::test_get_results	✅	21ms
tests/explorer/scheduler_test.py::SchedulerTest::test_scheduler_all_methods	✅	15ms
tests/explorer/scheduler_test.py::SchedulerTest::test_scheduler_restart_after_stop	✅	9ms
tests/explorer/scheduler_test.py::SchedulerTest::test_split_tasks	✅	8ms
tests/explorer/scheduler_test.py::SchedulerTest::test_wait_all	✅	8ms
tests/explorer/scheduler_test.py::SchedulerTest::test_wait_all_timeout_with_multi_batch	✅	14ms
tests/explorer/workflow_test.py::WorkflowTest::test_gsm8k_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_boxed_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_complex_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_fraction_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_rm_gallery_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_workflow_resettable	✅	1ms
tests/trainer/trainer_test.py::BaseTrainerCase::test_trainer	✅	1ms
tests/trainer/trainer_test.py::TestTrainerCountdown::test_trainer	✅	241ms
tests/trainer/trainer_test.py::TestStepAheadAsyncRL::test_trainer	✅	96ms
tests/trainer/trainer_test.py::TestTrainerGSM8K::test_trainer	✅	64ms
tests/trainer/trainer_test.py::TestTrainerSFTWarmupGSM8K::test_trainer	✅	98ms
tests/trainer/trainer_test.py::TestTrainerDPO::test_trainer	✅	45ms
tests/trainer/trainer_test.py::TestFullyAsyncMode::test_fully_async_mode_0_queue	✅	95ms
tests/trainer/trainer_test.py::TestFullyAsyncMode::test_fully_async_mode_1_priority_queue	✅	102ms
tests/utils/plugin_test.py::TestPluginLoader::test_load_plugins	✅	5ms

Github Test Reporter by CTRF 💚

trinity/common/rewards/dapo_reward.py

trinity/common/workflows/dapo_workflow.py

trinity/common/workflows/workflow.py

yanxi-chen

lgtm

hiyuchang · 2025-07-16T08:29:30Z

/run-unittest

github-actions · 2025-07-16T08:52:25Z

Summary

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Other ❓	Flaky 🍂	Duration ⏱️
51	50	1	0	0	0	1.3s

Failed Tests

Failed Tests ❌	Fail Message
❌ tests/explorer/workflow_test.py::WorkflowTest::test_math_boxed_workflow	The test failed in the call phase

Tests

Test Name	Status	Duration
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_dpo_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_mix_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_opmd_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_ppo_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_sft_policy_loss	✅	1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_buffer	✅	3ms
tests/buffer/file_test.py::TestFileBuffer::test_file_reader	✅	1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_writer	✅	1ms
tests/buffer/queue_test.py::TestQueueBuffer::test_priority_queue_buffer_reuse	✅	6ms
tests/buffer/queue_test.py::TestQueueBuffer::test_priority_queue_capacity	✅	2ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer_0_queue	✅	4ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer_1_priority_queue	✅	4ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer_capacity	✅	4ms
tests/buffer/sql_test.py::TestSQLBuffer::test_create_sql_buffer	✅	5ms
tests/common/config_test.py::TestConfig::test_all_examples_are_valid	✅	1ms
tests/common/config_test.py::TestConfig::test_load_default_config	✅	4ms
tests/common/experience_test.py::TestExperienceConversion::test_batch_conversion	✅	1ms
tests/common/experience_test.py::TestExperienceConversion::test_experience_model_experience_conversion	✅	1ms
tests/common/vllm_test.py::TestModelWrapperSyncV0::test_generate	✅	44ms
tests/common/vllm_test.py::TestModelWrapperAsyncV0::test_generate	✅	43ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV0::test_generate	✅	54ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV1::test_generate	✅	56ms
tests/common/vllm_test.py::TestModelWrapperAsyncV1::test_generate	✅	41ms
tests/common/vllm_test.py::TestAPIServer::test_api	✅	28ms
tests/common/vllm_test.py::TestTokenizer::test_assistant_token_mask	✅	1ms
tests/explorer/explorer_test.py::BaseExplorerCase::test_explorer	✅	1ms
tests/explorer/explorer_test.py::TestExplorerCountdownEval::test_explorer	✅	92ms
tests/explorer/explorer_test.py::TestExplorerCountdownNoEval::test_explorer	✅	99ms
tests/explorer/scheduler_test.py::SchedulerTest::test_concurrent_operations	✅	5ms
tests/explorer/scheduler_test.py::SchedulerTest::test_get_results	✅	21ms
tests/explorer/scheduler_test.py::SchedulerTest::test_scheduler_all_methods	✅	15ms
tests/explorer/scheduler_test.py::SchedulerTest::test_scheduler_restart_after_stop	✅	9ms
tests/explorer/scheduler_test.py::SchedulerTest::test_split_tasks	✅	8ms
tests/explorer/scheduler_test.py::SchedulerTest::test_wait_all	✅	8ms
tests/explorer/scheduler_test.py::SchedulerTest::test_wait_all_timeout_with_multi_batch	✅	14ms
tests/explorer/workflow_test.py::WorkflowTest::test_gsm8k_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_boxed_workflow	❌	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_complex_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_fraction_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_rm_gallery_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_workflow_resettable	✅	1ms
tests/trainer/trainer_test.py::BaseTrainerCase::test_trainer	✅	1ms
tests/trainer/trainer_test.py::TestTrainerCountdown::test_trainer	✅	244ms
tests/trainer/trainer_test.py::TestStepAheadAsyncRL::test_trainer	✅	96ms
tests/trainer/trainer_test.py::TestTrainerGSM8K::test_trainer	✅	61ms
tests/trainer/trainer_test.py::TestTrainerSFTWarmupGSM8K::test_trainer	✅	92ms
tests/trainer/trainer_test.py::TestTrainerDPO::test_trainer	✅	44ms
tests/trainer/trainer_test.py::TestFullyAsyncMode::test_fully_async_mode_0_queue	✅	98ms
tests/trainer/trainer_test.py::TestFullyAsyncMode::test_fully_async_mode_1_priority_queue	✅	104ms
tests/utils/plugin_test.py::TestPluginLoader::test_load_plugins	✅	5ms

Github Test Reporter by CTRF 💚

hiyuchang · 2025-07-16T09:13:14Z

/run-unittest

github-actions · 2025-07-16T09:36:22Z

Summary

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Other ❓	Flaky 🍂	Duration ⏱️
51	51	0	0	0	0	1.3s

Tests

Test Name	Status	Duration
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_dpo_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_mix_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_opmd_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_ppo_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_sft_policy_loss	✅	1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_buffer	✅	3ms
tests/buffer/file_test.py::TestFileBuffer::test_file_reader	✅	1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_writer	✅	1ms
tests/buffer/queue_test.py::TestQueueBuffer::test_priority_queue_buffer_reuse	✅	6ms
tests/buffer/queue_test.py::TestQueueBuffer::test_priority_queue_capacity	✅	2ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer_0_queue	✅	4ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer_1_priority_queue	✅	4ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer_capacity	✅	4ms
tests/buffer/sql_test.py::TestSQLBuffer::test_create_sql_buffer	✅	4ms
tests/common/config_test.py::TestConfig::test_all_examples_are_valid	✅	1ms
tests/common/config_test.py::TestConfig::test_load_default_config	✅	4ms
tests/common/experience_test.py::TestExperienceConversion::test_batch_conversion	✅	1ms
tests/common/experience_test.py::TestExperienceConversion::test_experience_model_experience_conversion	✅	1ms
tests/common/vllm_test.py::TestModelWrapperSyncV0::test_generate	✅	45ms
tests/common/vllm_test.py::TestModelWrapperAsyncV0::test_generate	✅	42ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV0::test_generate	✅	54ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV1::test_generate	✅	56ms
tests/common/vllm_test.py::TestModelWrapperAsyncV1::test_generate	✅	41ms
tests/common/vllm_test.py::TestAPIServer::test_api	✅	28ms
tests/common/vllm_test.py::TestTokenizer::test_assistant_token_mask	✅	1ms
tests/explorer/explorer_test.py::BaseExplorerCase::test_explorer	✅	1ms
tests/explorer/explorer_test.py::TestExplorerCountdownEval::test_explorer	✅	96ms
tests/explorer/explorer_test.py::TestExplorerCountdownNoEval::test_explorer	✅	107ms
tests/explorer/scheduler_test.py::SchedulerTest::test_concurrent_operations	✅	5ms
tests/explorer/scheduler_test.py::SchedulerTest::test_get_results	✅	21ms
tests/explorer/scheduler_test.py::SchedulerTest::test_scheduler_all_methods	✅	15ms
tests/explorer/scheduler_test.py::SchedulerTest::test_scheduler_restart_after_stop	✅	9ms
tests/explorer/scheduler_test.py::SchedulerTest::test_split_tasks	✅	8ms
tests/explorer/scheduler_test.py::SchedulerTest::test_wait_all	✅	8ms
tests/explorer/scheduler_test.py::SchedulerTest::test_wait_all_timeout_with_multi_batch	✅	14ms
tests/explorer/workflow_test.py::WorkflowTest::test_gsm8k_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_boxed_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_complex_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_fraction_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_rm_gallery_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_workflow_resettable	✅	1ms
tests/trainer/trainer_test.py::BaseTrainerCase::test_trainer	✅	1ms
tests/trainer/trainer_test.py::TestTrainerCountdown::test_trainer	✅	238ms
tests/trainer/trainer_test.py::TestStepAheadAsyncRL::test_trainer	✅	97ms
tests/trainer/trainer_test.py::TestTrainerGSM8K::test_trainer	✅	66ms
tests/trainer/trainer_test.py::TestTrainerSFTWarmupGSM8K::test_trainer	✅	87ms
tests/trainer/trainer_test.py::TestTrainerDPO::test_trainer	✅	46ms
tests/trainer/trainer_test.py::TestFullyAsyncMode::test_fully_async_mode_0_queue	✅	103ms
tests/trainer/trainer_test.py::TestFullyAsyncMode::test_fully_async_mode_1_priority_queue	✅	102ms
tests/utils/plugin_test.py::TestPluginLoader::test_load_plugins	✅	5ms

Github Test Reporter by CTRF 💚

trinity/common/rewards/dapo_reward.py

add dapo workflow and reward

b8c262a

fix bug

b9076b7

garyzhang99 reviewed Jul 10, 2025

View reviewed changes

trinity/common/workflows/dapo_workflow.py Outdated Show resolved Hide resolved

hiyuchang added 5 commits July 14, 2025 10:20

Merge branch 'main' into feat/len_punish

da57574

add base model mode for dapo_workflow

889b36c

Merge branch 'main' into feat/len_punish

a28c310

fix dapo config

880c5b0

Merge branch 'main' into feat/len_punish

fd39fe4

fix typo

c314a11

yanxi-chen reviewed Jul 15, 2025

View reviewed changes

trinity/common/rewards/dapo_reward.py Show resolved Hide resolved

yanxi-chen reviewed Jul 15, 2025

View reviewed changes

trinity/common/rewards/dapo_reward.py Outdated Show resolved Hide resolved

yanxi-chen reviewed Jul 15, 2025

View reviewed changes

trinity/common/workflows/dapo_workflow.py Outdated Show resolved Hide resolved

yanxi-chen reviewed Jul 15, 2025

View reviewed changes

trinity/common/workflows/workflow.py Outdated Show resolved Hide resolved

fix comments

e1c7622

yanxi-chen approved these changes Jul 15, 2025

View reviewed changes

mv dapo workflow to boxed workflo

7621887

hiyuchang changed the title ~~Add dapo workflow and reward~~ Add dapo reward Jul 16, 2025

fix unittest

ca04b7a

pan-x-c reviewed Jul 16, 2025

View reviewed changes

trinity/common/rewards/dapo_reward.py Outdated Show resolved Hide resolved

hiyuchang added 2 commits July 16, 2025 19:24

fix typo

7d96213

fix typo

3239c73

pan-x-c approved these changes Jul 16, 2025

View reviewed changes

pan-x-c merged commit a7c142f into modelscope:main Jul 16, 2025
1 check passed

Add dapo reward #114

Add dapo reward #114

Uh oh!

Conversation

hiyuchang commented Jul 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Uh oh!

hiyuchang commented Jul 10, 2025

Uh oh!

github-actions bot commented Jul 10, 2025

Summary

Tests

Uh oh!

chenyushuo commented Jul 10, 2025

Uh oh!

github-actions bot commented Jul 10, 2025

Summary

Tests

Uh oh!

hiyuchang commented Jul 10, 2025

Uh oh!

github-actions bot commented Jul 10, 2025

Summary

Tests

Uh oh!

Uh oh!

hiyuchang commented Jul 14, 2025

Uh oh!

github-actions bot commented Jul 14, 2025

Summary

Failed Tests

Tests

Uh oh!

hiyuchang commented Jul 14, 2025

Uh oh!

github-actions bot commented Jul 14, 2025

Summary

Tests

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yanxi-chen left a comment

Choose a reason for hiding this comment

Uh oh!

hiyuchang commented Jul 16, 2025

Uh oh!

github-actions bot commented Jul 16, 2025

Summary

Failed Tests

Tests

Uh oh!

hiyuchang commented Jul 16, 2025

Uh oh!

github-actions bot commented Jul 16, 2025

Summary

Tests

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

hiyuchang commented Jul 10, 2025 •

edited

Loading