Trinity-RFT v0.2.0 #72

pan-x-c · 2025-06-09T05:22:19Z

Description

Refactor Algorithm-related modules, see Refactor Algorithm-Related Code for Better Maintainability and Extensibility #59 for details
Propose an SFT/GRPO mixed algorithm.
Unify Sync / Async RL by sync_interval, and support one-step async pipeline
Refactor data processor module, and support preprocess input taskset
Update vLLM to v0.9.1, verl to 0.4.0

Checklist

Please check the following items before code is ready to be reviewed.

Code has passed all tests
Docstrings have been added/updated in Google Style
Documentation has been updated
Code is ready for review

Co-authored-by: weijie <[email protected]>

pan-x-c · 2025-06-11T10:26:14Z

/run-unittest

github-actions · 2025-06-11T10:42:58Z

Summary

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Pending ⏳	Other ❓	Flaky 🍂	Duration ⏱️
29	28	1	0	0	0	0	967ms

Failed Tests

Failed Tests ❌	Fail Message
❌ tests/trainer/trainer_test.py::TestTrainerGSM8KWithSFT::test_trainer	The test failed in the call phase due to an exception

Flaky Tests

No flaky tests ✨

Skipped

No skipped tests ✨

Tests

Test Name	Status	Duration
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer	✅	4ms
tests/buffer/sql_test.py::TestSQLBuffer::test_create_sql_buffer	✅	2ms
tests/common/config_test.py::TestConfig::test_all_examples_are_valid	✅	1ms
tests/common/config_test.py::TestConfig::test_load_default_config	✅	5ms
tests/common/experience_test.py::TestExperienceConversion::test_batch_conversion	✅	1ms
tests/common/experience_test.py::TestExperienceConversion::test_experience_model_experience_conversion	✅	1ms
tests/common/vllm_test.py::TestModelWrapperSyncV0::test_generate	✅	44ms
tests/common/vllm_test.py::TestModelWrapperAsyncV0::test_generate	✅	43ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV0::test_generate	✅	53ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV1::test_generate	✅	53ms
tests/common/vllm_test.py::TestModelWrapperAsyncV1::test_generate	✅	39ms
tests/common/vllm_test.py::TestAPIServer::test_api	✅	21ms
tests/common/vllm_test.py::TestTokenizer::test_assistant_token_mask	✅	1ms
tests/explorer/explorer_test.py::BaseExplorerCase::test_explorer	✅	1ms
tests/explorer/explorer_test.py::TestExplorerCountdownEval::test_explorer	✅	108ms
tests/explorer/explorer_test.py::TestExplorerCountdownNoEval::test_explorer	✅	139ms
tests/explorer/runner_pool_test.py::RunnerPoolTest::test_runner_pool	✅	20ms
tests/explorer/runner_pool_test.py::RunnerPoolTest::test_runner_pool_with_auxiliary_models	✅	3ms
tests/explorer/workflow_test.py::WorkflowTest::test_gsm8k_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_complex_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_fraction_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_workflow_resettable	✅	1ms
tests/trainer/trainer_test.py::BaseTrainerCase::test_trainer	✅	1ms
tests/trainer/trainer_test.py::TestTrainerCountdown::test_trainer	✅	266ms
tests/trainer/trainer_test.py::TestTrainerGSM8K::test_trainer	✅	59ms
tests/trainer/trainer_test.py::TestTrainerGSM8KWithSFT::test_trainer	❌	27ms
tests/trainer/trainer_test.py::TestTrainerDPO::test_trainer	✅	61ms
tests/utils/plugin_test.py::TestPluginLoader::test_load_plugins	✅	4ms

Github Test Reporter by CTRF 💚

pan-x-c · 2025-06-11T11:13:38Z

/run-unittest

github-actions · 2025-06-11T11:31:26Z

Summary

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Pending ⏳	Other ❓	Flaky 🍂	Duration ⏱️
29	29	0	0	0	0	0	1.0s

Failed Tests

No failed tests ✨

Flaky Tests

No flaky tests ✨

Skipped

No skipped tests ✨

Tests

Test Name	Status	Duration
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer	✅	4ms
tests/buffer/sql_test.py::TestSQLBuffer::test_create_sql_buffer	✅	2ms
tests/common/config_test.py::TestConfig::test_all_examples_are_valid	✅	1ms
tests/common/config_test.py::TestConfig::test_load_default_config	✅	5ms
tests/common/experience_test.py::TestExperienceConversion::test_batch_conversion	✅	1ms
tests/common/experience_test.py::TestExperienceConversion::test_experience_model_experience_conversion	✅	1ms
tests/common/vllm_test.py::TestModelWrapperSyncV0::test_generate	✅	45ms
tests/common/vllm_test.py::TestModelWrapperAsyncV0::test_generate	✅	43ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV0::test_generate	✅	52ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV1::test_generate	✅	53ms
tests/common/vllm_test.py::TestModelWrapperAsyncV1::test_generate	✅	39ms
tests/common/vllm_test.py::TestAPIServer::test_api	✅	21ms
tests/common/vllm_test.py::TestTokenizer::test_assistant_token_mask	✅	1ms
tests/explorer/explorer_test.py::BaseExplorerCase::test_explorer	✅	1ms
tests/explorer/explorer_test.py::TestExplorerCountdownEval::test_explorer	✅	110ms
tests/explorer/explorer_test.py::TestExplorerCountdownNoEval::test_explorer	✅	145ms
tests/explorer/runner_pool_test.py::RunnerPoolTest::test_runner_pool	✅	20ms
tests/explorer/runner_pool_test.py::RunnerPoolTest::test_runner_pool_with_auxiliary_models	✅	3ms
tests/explorer/workflow_test.py::WorkflowTest::test_gsm8k_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_complex_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_fraction_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_workflow_resettable	✅	1ms
tests/trainer/trainer_test.py::BaseTrainerCase::test_trainer	✅	1ms
tests/trainer/trainer_test.py::TestTrainerCountdown::test_trainer	✅	294ms
tests/trainer/trainer_test.py::TestTrainerGSM8K::test_trainer	✅	59ms
tests/trainer/trainer_test.py::TestTrainerGSM8KWithSFT::test_trainer	✅	63ms
tests/trainer/trainer_test.py::TestTrainerDPO::test_trainer	✅	54ms
tests/utils/plugin_test.py::TestPluginLoader::test_load_plugins	✅	4ms

Github Test Reporter by CTRF 💚

pan-x-c · 2025-06-18T03:46:03Z

/run-unittest

github-actions · 2025-06-18T04:03:14Z

Summary

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Pending ⏳	Other ❓	Flaky 🍂	Duration ⏱️
36	34	2	0	0	0	0	997ms

Failed Tests

Failed Tests ❌	Fail Message
❌ tests/trainer/trainer_test.py::TestTrainerGSM8KWithSFT::test_trainer	The test failed in the call phase due to an exception
❌ tests/trainer/trainer_test.py::TestTrainerDPO::test_trainer	The test failed in the call phase due to an exception

Flaky Tests

No flaky tests ✨

Skipped

No skipped tests ✨

Tests

Test Name	Status	Duration
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_dpo_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_mix_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_opmd_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_ppo_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_sft_policy_loss	✅	1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_reader	✅	1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_writer	✅	3ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer	✅	2ms
tests/buffer/sql_test.py::TestSQLBuffer::test_create_sql_buffer	✅	3ms
tests/common/config_test.py::TestConfig::test_all_examples_are_valid	✅	1ms
tests/common/config_test.py::TestConfig::test_load_default_config	✅	5ms
tests/common/experience_test.py::TestExperienceConversion::test_batch_conversion	✅	1ms
tests/common/experience_test.py::TestExperienceConversion::test_experience_model_experience_conversion	✅	1ms
tests/common/vllm_test.py::TestModelWrapperSyncV0::test_generate	✅	44ms
tests/common/vllm_test.py::TestModelWrapperAsyncV0::test_generate	✅	43ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV0::test_generate	✅	52ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV1::test_generate	✅	52ms
tests/common/vllm_test.py::TestModelWrapperAsyncV1::test_generate	✅	38ms
tests/common/vllm_test.py::TestAPIServer::test_api	✅	21ms
tests/common/vllm_test.py::TestTokenizer::test_assistant_token_mask	✅	1ms
tests/explorer/explorer_test.py::BaseExplorerCase::test_explorer	✅	1ms
tests/explorer/explorer_test.py::TestExplorerCountdownEval::test_explorer	✅	121ms
tests/explorer/explorer_test.py::TestExplorerCountdownNoEval::test_explorer	✅	142ms
tests/explorer/runner_pool_test.py::RunnerPoolTest::test_runner_pool	✅	22ms
tests/explorer/runner_pool_test.py::RunnerPoolTest::test_runner_pool_with_auxiliary_models	✅	4ms
tests/explorer/workflow_test.py::WorkflowTest::test_gsm8k_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_complex_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_fraction_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_workflow_resettable	✅	1ms
tests/trainer/trainer_test.py::BaseTrainerCase::test_trainer	✅	1ms
tests/trainer/trainer_test.py::TestTrainerCountdown::test_trainer	✅	338ms
tests/trainer/trainer_test.py::TestTrainerGSM8K::test_trainer	✅	67ms
tests/trainer/trainer_test.py::TestTrainerGSM8KWithSFT::test_trainer	❌	6ms
tests/trainer/trainer_test.py::TestTrainerDPO::test_trainer	❌	12ms
tests/utils/plugin_test.py::TestPluginLoader::test_load_plugins	✅	4ms

Github Test Reporter by CTRF 💚

pan-x-c · 2025-06-18T04:25:56Z

/run-unittest

github-actions · 2025-06-18T04:44:09Z

Summary

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Pending ⏳	Other ❓	Flaky 🍂	Duration ⏱️
36	35	1	0	0	0	0	1.1s

Failed Tests

Failed Tests ❌	Fail Message
❌ tests/trainer/trainer_test.py::TestTrainerGSM8KWithSFT::test_trainer	The test failed in the call phase due to an exception

Flaky Tests

No flaky tests ✨

Skipped

No skipped tests ✨

Tests

Test Name	Status	Duration
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_dpo_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_mix_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_opmd_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_ppo_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_sft_policy_loss	✅	1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_reader	✅	1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_writer	✅	3ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer	✅	2ms
tests/buffer/sql_test.py::TestSQLBuffer::test_create_sql_buffer	✅	3ms
tests/common/config_test.py::TestConfig::test_all_examples_are_valid	✅	1ms
tests/common/config_test.py::TestConfig::test_load_default_config	✅	5ms
tests/common/experience_test.py::TestExperienceConversion::test_batch_conversion	✅	1ms
tests/common/experience_test.py::TestExperienceConversion::test_experience_model_experience_conversion	✅	1ms
tests/common/vllm_test.py::TestModelWrapperSyncV0::test_generate	✅	44ms
tests/common/vllm_test.py::TestModelWrapperAsyncV0::test_generate	✅	44ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV0::test_generate	✅	52ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV1::test_generate	✅	53ms
tests/common/vllm_test.py::TestModelWrapperAsyncV1::test_generate	✅	39ms
tests/common/vllm_test.py::TestAPIServer::test_api	✅	22ms
tests/common/vllm_test.py::TestTokenizer::test_assistant_token_mask	✅	1ms
tests/explorer/explorer_test.py::BaseExplorerCase::test_explorer	✅	1ms
tests/explorer/explorer_test.py::TestExplorerCountdownEval::test_explorer	✅	126ms
tests/explorer/explorer_test.py::TestExplorerCountdownNoEval::test_explorer	✅	143ms
tests/explorer/runner_pool_test.py::RunnerPoolTest::test_runner_pool	✅	22ms
tests/explorer/runner_pool_test.py::RunnerPoolTest::test_runner_pool_with_auxiliary_models	✅	4ms
tests/explorer/workflow_test.py::WorkflowTest::test_gsm8k_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_complex_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_fraction_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_workflow_resettable	✅	1ms
tests/trainer/trainer_test.py::BaseTrainerCase::test_trainer	✅	1ms
tests/trainer/trainer_test.py::TestTrainerCountdown::test_trainer	✅	318ms
tests/trainer/trainer_test.py::TestTrainerGSM8K::test_trainer	✅	70ms
tests/trainer/trainer_test.py::TestTrainerGSM8KWithSFT::test_trainer	❌	21ms
tests/trainer/trainer_test.py::TestTrainerDPO::test_trainer	✅	61ms
tests/utils/plugin_test.py::TestPluginLoader::test_load_plugins	✅	4ms

Github Test Reporter by CTRF 💚

pan-x-c · 2025-06-18T05:52:10Z

/run-unittest

github-actions · 2025-06-18T06:10:49Z

Summary

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Pending ⏳	Other ❓	Flaky 🍂	Duration ⏱️
36	36	0	0	0	0	0	1.1s

Failed Tests

No failed tests ✨

Flaky Tests

No flaky tests ✨

Skipped

No skipped tests ✨

Tests

Test Name	Status	Duration
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_dpo_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_mix_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_opmd_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_ppo_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_sft_policy_loss	✅	1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_reader	✅	1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_writer	✅	3ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer	✅	2ms
tests/buffer/sql_test.py::TestSQLBuffer::test_create_sql_buffer	✅	3ms
tests/common/config_test.py::TestConfig::test_all_examples_are_valid	✅	1ms
tests/common/config_test.py::TestConfig::test_load_default_config	✅	6ms
tests/common/experience_test.py::TestExperienceConversion::test_batch_conversion	✅	1ms
tests/common/experience_test.py::TestExperienceConversion::test_experience_model_experience_conversion	✅	1ms
tests/common/vllm_test.py::TestModelWrapperSyncV0::test_generate	✅	41ms
tests/common/vllm_test.py::TestModelWrapperAsyncV0::test_generate	✅	40ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV0::test_generate	✅	51ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV1::test_generate	✅	51ms
tests/common/vllm_test.py::TestModelWrapperAsyncV1::test_generate	✅	37ms
tests/common/vllm_test.py::TestAPIServer::test_api	✅	24ms
tests/common/vllm_test.py::TestTokenizer::test_assistant_token_mask	✅	1ms
tests/explorer/explorer_test.py::BaseExplorerCase::test_explorer	✅	1ms
tests/explorer/explorer_test.py::TestExplorerCountdownEval::test_explorer	✅	95ms
tests/explorer/explorer_test.py::TestExplorerCountdownNoEval::test_explorer	✅	102ms
tests/explorer/runner_pool_test.py::RunnerPoolTest::test_runner_pool	✅	22ms
tests/explorer/runner_pool_test.py::RunnerPoolTest::test_runner_pool_with_auxiliary_models	✅	4ms
tests/explorer/workflow_test.py::WorkflowTest::test_gsm8k_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_complex_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_fraction_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_workflow_resettable	✅	1ms
tests/trainer/trainer_test.py::BaseTrainerCase::test_trainer	✅	1ms
tests/trainer/trainer_test.py::TestTrainerCountdown::test_trainer	✅	370ms
tests/trainer/trainer_test.py::TestTrainerGSM8K::test_trainer	✅	64ms
tests/trainer/trainer_test.py::TestTrainerGSM8KWithSFT::test_trainer	✅	83ms
tests/trainer/trainer_test.py::TestTrainerDPO::test_trainer	✅	62ms
tests/utils/plugin_test.py::TestPluginLoader::test_load_plugins	✅	4ms

Github Test Reporter by CTRF 💚

pan-x-c · 2025-06-18T11:09:07Z

/run-unittest

github-actions · 2025-06-18T14:28:35Z

Summary

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Pending ⏳	Other ❓	Flaky 🍂	Duration ⏱️
36	36	0	0	0	0	0	11.9s

Failed Tests

No failed tests ✨

Flaky Tests

No flaky tests ✨

Skipped

No skipped tests ✨

Tests

Test Name	Status	Duration
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_dpo_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_mix_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_opmd_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_ppo_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_sft_policy_loss	✅	1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_reader	✅	1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_writer	✅	2ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer	✅	2ms
tests/buffer/sql_test.py::TestSQLBuffer::test_create_sql_buffer	✅	3ms
tests/common/config_test.py::TestConfig::test_all_examples_are_valid	✅	1ms
tests/common/config_test.py::TestConfig::test_load_default_config	✅	5ms
tests/common/experience_test.py::TestExperienceConversion::test_batch_conversion	✅	1ms
tests/common/experience_test.py::TestExperienceConversion::test_experience_model_experience_conversion	✅	1ms
tests/common/vllm_test.py::TestModelWrapperSyncV0::test_generate	✅	42ms
tests/common/vllm_test.py::TestModelWrapperAsyncV0::test_generate	✅	40ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV0::test_generate	✅	51ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV1::test_generate	✅	51ms
tests/common/vllm_test.py::TestModelWrapperAsyncV1::test_generate	✅	38ms
tests/common/vllm_test.py::TestAPIServer::test_api	✅	24ms
tests/common/vllm_test.py::TestTokenizer::test_assistant_token_mask	✅	1ms
tests/explorer/explorer_test.py::BaseExplorerCase::test_explorer	✅	1ms
tests/explorer/explorer_test.py::TestExplorerCountdownEval::test_explorer	✅	93ms
tests/explorer/explorer_test.py::TestExplorerCountdownNoEval::test_explorer	✅	110ms
tests/explorer/runner_pool_test.py::RunnerPoolTest::test_runner_pool	✅	22ms
tests/explorer/runner_pool_test.py::RunnerPoolTest::test_runner_pool_with_auxiliary_models	✅	4ms
tests/explorer/workflow_test.py::WorkflowTest::test_gsm8k_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_complex_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_fraction_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_workflow_resettable	✅	1ms
tests/trainer/trainer_test.py::BaseTrainerCase::test_trainer	✅	1ms
tests/trainer/trainer_test.py::TestTrainerCountdown::test_trainer	✅	11.2s
tests/trainer/trainer_test.py::TestTrainerGSM8K::test_trainer	✅	72ms
tests/trainer/trainer_test.py::TestTrainerGSM8KWithSFT::test_trainer	✅	81ms
tests/trainer/trainer_test.py::TestTrainerDPO::test_trainer	✅	63ms
tests/utils/plugin_test.py::TestPluginLoader::test_load_plugins	✅	4ms

Github Test Reporter by CTRF 💚

pan-x-c · 2025-06-19T02:28:25Z

/run-unittest

github-actions · 2025-06-19T02:46:20Z

Summary

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Pending ⏳	Other ❓	Flaky 🍂	Duration ⏱️
36	36	0	0	0	0	0	1.0s

Failed Tests

No failed tests ✨

Flaky Tests

No flaky tests ✨

Skipped

No skipped tests ✨

Tests

Test Name	Status	Duration
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_dpo_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_mix_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_opmd_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_ppo_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_sft_policy_loss	✅	1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_reader	✅	1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_writer	✅	3ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer	✅	2ms
tests/buffer/sql_test.py::TestSQLBuffer::test_create_sql_buffer	✅	3ms
tests/common/config_test.py::TestConfig::test_all_examples_are_valid	✅	1ms
tests/common/config_test.py::TestConfig::test_load_default_config	✅	5ms
tests/common/experience_test.py::TestExperienceConversion::test_batch_conversion	✅	1ms
tests/common/experience_test.py::TestExperienceConversion::test_experience_model_experience_conversion	✅	1ms
tests/common/vllm_test.py::TestModelWrapperSyncV0::test_generate	✅	41ms
tests/common/vllm_test.py::TestModelWrapperAsyncV0::test_generate	✅	41ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV0::test_generate	✅	50ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV1::test_generate	✅	51ms
tests/common/vllm_test.py::TestModelWrapperAsyncV1::test_generate	✅	38ms
tests/common/vllm_test.py::TestAPIServer::test_api	✅	24ms
tests/common/vllm_test.py::TestTokenizer::test_assistant_token_mask	✅	1ms
tests/explorer/explorer_test.py::BaseExplorerCase::test_explorer	✅	1ms
tests/explorer/explorer_test.py::TestExplorerCountdownEval::test_explorer	✅	101ms
tests/explorer/explorer_test.py::TestExplorerCountdownNoEval::test_explorer	✅	117ms
tests/explorer/runner_pool_test.py::RunnerPoolTest::test_runner_pool	✅	22ms
tests/explorer/runner_pool_test.py::RunnerPoolTest::test_runner_pool_with_auxiliary_models	✅	4ms
tests/explorer/workflow_test.py::WorkflowTest::test_gsm8k_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_complex_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_fraction_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_workflow_resettable	✅	1ms
tests/trainer/trainer_test.py::BaseTrainerCase::test_trainer	✅	1ms
tests/trainer/trainer_test.py::TestTrainerCountdown::test_trainer	✅	308ms
tests/trainer/trainer_test.py::TestTrainerGSM8K::test_trainer	✅	65ms
tests/trainer/trainer_test.py::TestTrainerGSM8KWithSFT::test_trainer	✅	81ms
tests/trainer/trainer_test.py::TestTrainerDPO::test_trainer	✅	62ms
tests/utils/plugin_test.py::TestPluginLoader::test_load_plugins	✅	4ms

Github Test Reporter by CTRF 💚

pan-x-c · 2025-06-20T09:49:30Z

/run-unittest

github-actions · 2025-06-20T10:08:17Z

Summary

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Pending ⏳	Other ❓	Flaky 🍂	Duration ⏱️
39	39	0	0	0	0	0	1.1s

Failed Tests

No failed tests ✨

Flaky Tests

No flaky tests ✨

Skipped

No skipped tests ✨

Tests

Test Name	Status	Duration
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_dpo_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_mix_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_opmd_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_ppo_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_sft_policy_loss	✅	1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_buffer	✅	3ms
tests/buffer/file_test.py::TestFileBuffer::test_file_reader	✅	1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_writer	✅	1ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer	✅	2ms
tests/buffer/sql_test.py::TestSQLBuffer::test_create_sql_buffer	✅	3ms
tests/common/config_test.py::TestConfig::test_all_examples_are_valid	✅	1ms
tests/common/config_test.py::TestConfig::test_load_default_config	✅	5ms
tests/common/experience_test.py::TestExperienceConversion::test_batch_conversion	✅	1ms
tests/common/experience_test.py::TestExperienceConversion::test_experience_model_experience_conversion	✅	1ms
tests/common/vllm_test.py::TestModelWrapperSyncV0::test_generate	✅	42ms
tests/common/vllm_test.py::TestModelWrapperAsyncV0::test_generate	✅	41ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV0::test_generate	✅	51ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV1::test_generate	✅	50ms
tests/common/vllm_test.py::TestModelWrapperAsyncV1::test_generate	✅	38ms
tests/common/vllm_test.py::TestAPIServer::test_api	✅	24ms
tests/common/vllm_test.py::TestTokenizer::test_assistant_token_mask	✅	1ms
tests/explorer/explorer_test.py::BaseExplorerCase::test_explorer	✅	1ms
tests/explorer/explorer_test.py::TestExplorerCountdownEval::test_explorer	✅	90ms
tests/explorer/explorer_test.py::TestExplorerCountdownNoEval::test_explorer	✅	115ms
tests/explorer/runner_pool_test.py::RunnerPoolTest::test_runner_pool	✅	22ms
tests/explorer/runner_pool_test.py::RunnerPoolTest::test_runner_pool_with_auxiliary_models	✅	4ms
tests/explorer/workflow_test.py::WorkflowTest::test_gsm8k_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_boxed_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_complex_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_fraction_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_workflow_resettable	✅	1ms
tests/trainer/trainer_test.py::BaseTrainerCase::test_trainer	✅	1ms
tests/trainer/trainer_test.py::TestTrainerCountdown::test_trainer	✅	274ms
tests/trainer/trainer_test.py::TestStepAheadAsyncRL::test_trainer	✅	115ms
tests/trainer/trainer_test.py::TestTrainerGSM8K::test_trainer	✅	65ms
tests/trainer/trainer_test.py::TestTrainerSFTWarmupGSM8K::test_trainer	✅	68ms
tests/trainer/trainer_test.py::TestTrainerDPO::test_trainer	✅	53ms
tests/utils/plugin_test.py::TestPluginLoader::test_load_plugins	✅	4ms

Github Test Reporter by CTRF 💚

yanxi-chen

lgtm

pan-x-c and others added 6 commits May 28, 2025 13:55

Init Algorithm Module (#58)

d7c43fe

Add Policy Loss Functions (#62)

5cd6cb6

Refactor advantage computation, and delete RayPPOTrainer.fit (#61)

fe217aa

Add unittest && bug fix (#65)

9d582e8

Add KL/Entorpy Fn (#64)

732d801

Refactor advantage computation (cont.) (#68)

2d8f0c1

pan-x-c linked an issue Jun 9, 2025 that may be closed by this pull request

Refactor Algorithm-Related Code for Better Maintainability and Extensibility #59

Closed

chenyushuo and others added 4 commits June 10, 2025 20:55

Refactor train step (#69)

fec7f3c

Fix EntropyLossFn (#77)

48f596a

Co-authored-by: weijie <[email protected]>

Fix Conflicts with main (#75)

9e72996

merge main

21ddf5e

fix entropy lss

3c759d9

pan-x-c and others added 7 commits June 12, 2025 15:41

Add Sample Strategy (#78)

dc8cb0c

Add doc for SFT (#81)

0e56607

merge verl 0.4.0 (#79)

aeabfe5

[Feature] Add MIX algorithm (#83)

b8d1faa

Refactor on select_keys (#84)

69ddbd0

Add guideline for adding new algorithm (#85)

a592af7

merge main

ab28d0c

fix file_reader

b7ea08f

pan-x-c added 2 commits June 18, 2025 13:24

update pyproject

eea4d85

update pyproject.toml

aedfa53

clean code

2a36e0e

fix checkpoint sync mode

5cb9ebe

chenyushuo and others added 8 commits June 19, 2025 14:45

Update config manager (#86)

f24db44

Update docs (#89)

c85d853

Refactor state_dict_meta init (#90)

7a1c526

Unify async/sync RL (#91)

99a772a

Support one-step ahead async RL (#93)

6f2d7c7

Refactor data module and support task pipeline in data processor (#92)

eddf4e4

Merge branch 'main' into algorithm_dev

7bc465c

bumping repository code version to v0.2.0.dev0

acf7788

pan-x-c changed the title ~~Refactor algorithm-related modules~~ Trinity-RFT v0.2.0 Jun 20, 2025

pan-x-c added this to the A tentative roadmap milestone Jun 20, 2025

yanxi-chen approved these changes Jun 20, 2025

View reviewed changes

pan-x-c merged commit d94967b into main Jun 20, 2025
3 checks passed

Trinity-RFT v0.2.0 #72

Trinity-RFT v0.2.0 #72

Uh oh!

Conversation

pan-x-c commented Jun 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Uh oh!

pan-x-c commented Jun 11, 2025

Uh oh!

github-actions bot commented Jun 11, 2025

Summary

Failed Tests

Flaky Tests

Skipped

Tests

Uh oh!

pan-x-c commented Jun 11, 2025

Uh oh!

github-actions bot commented Jun 11, 2025

Summary

Failed Tests

Flaky Tests

Skipped

Tests

Uh oh!

pan-x-c commented Jun 18, 2025

Uh oh!

github-actions bot commented Jun 18, 2025

Summary

Failed Tests

Flaky Tests

Skipped

Tests

Uh oh!

pan-x-c commented Jun 18, 2025

Uh oh!

github-actions bot commented Jun 18, 2025

Summary

Failed Tests

Flaky Tests

Skipped

Tests

Uh oh!

pan-x-c commented Jun 18, 2025

Uh oh!

github-actions bot commented Jun 18, 2025

Summary

Failed Tests

Flaky Tests

Skipped

Tests

Uh oh!

pan-x-c commented Jun 18, 2025

Uh oh!

github-actions bot commented Jun 18, 2025

Summary

Failed Tests

Flaky Tests

Skipped

Tests

Uh oh!

pan-x-c commented Jun 19, 2025

Uh oh!

github-actions bot commented Jun 19, 2025

Summary

Failed Tests

Flaky Tests

Skipped

Tests

Uh oh!

pan-x-c commented Jun 20, 2025

Uh oh!

github-actions bot commented Jun 20, 2025

Summary

Failed Tests

Flaky Tests

Skipped

Tests

pan-x-c commented Jun 9, 2025 •

edited

Loading