ModelWrapper automatically record Experience #123

pan-x-c · 2025-07-15T03:55:11Z

Description

Use ModelWrapper to record the model API call history and automatically generate Experience data from it.
This can reduce the difficulty of writing Workflows.

client = model_wrapper.get_openai_client()
# run some agent app with the client via client.chat.completions.create
exps = model_wrapper.extract_experience_from_history()

use async vllm model by default, and remove the sync one.
record generate/chat history
record openai.OpenAI.chat.completions.create history
generate Experience list from history data

Checklist

Please check the following items before code is ready to be reviewed.

Code has passed all tests
Docstrings have been added/updated in Google Style
Documentation has been updated
Code is ready for review

pan-x-c · 2025-07-16T13:07:17Z

/run-unittest

github-actions · 2025-07-16T13:35:40Z

Summary

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Other ❓	Flaky 🍂	Duration ⏱️
50	50	0	0	0	0	1.4s

Tests

Test Name	Status	Duration
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_dpo_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_mix_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_opmd_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_ppo_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_sft_policy_loss	✅	1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_buffer	✅	3ms
tests/buffer/file_test.py::TestFileBuffer::test_file_reader	✅	1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_writer	✅	2ms
tests/buffer/queue_test.py::TestQueueBuffer::test_priority_queue_buffer_reuse	✅	6ms
tests/buffer/queue_test.py::TestQueueBuffer::test_priority_queue_capacity	✅	2ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer_0_queue	✅	4ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer_1_priority_queue	✅	4ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer_capacity	✅	4ms
tests/buffer/sql_test.py::TestSQLBuffer::test_create_sql_buffer	✅	5ms
tests/common/config_test.py::TestConfig::test_all_examples_are_valid	✅	1ms
tests/common/config_test.py::TestConfig::test_load_default_config	✅	8ms
tests/common/experience_test.py::TestExperienceConversion::test_batch_conversion	✅	1ms
tests/common/experience_test.py::TestExperienceConversion::test_experience_model_experience_conversion	✅	1ms
tests/common/vllm_test.py::TestModelWrapperAsyncV0::test_generate	✅	44ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV0::test_generate	✅	54ms
tests/common/vllm_test.py::TestModelWrapperAsyncTPV1::test_generate	✅	56ms
tests/common/vllm_test.py::TestModelWrapperAsyncV1::test_generate	✅	42ms
tests/common/vllm_test.py::TestAPIServer::test_api	✅	27ms
tests/common/vllm_test.py::TestTokenizer::test_assistant_token_mask	✅	1ms
tests/explorer/explorer_test.py::BaseExplorerCase::test_explorer	✅	1ms
tests/explorer/explorer_test.py::TestExplorerCountdownEval::test_explorer	✅	89ms
tests/explorer/explorer_test.py::TestExplorerCountdownNoEval::test_explorer	✅	89ms
tests/explorer/scheduler_test.py::SchedulerTest::test_concurrent_operations	✅	5ms
tests/explorer/scheduler_test.py::SchedulerTest::test_get_results	✅	21ms
tests/explorer/scheduler_test.py::SchedulerTest::test_scheduler_all_methods	✅	15ms
tests/explorer/scheduler_test.py::SchedulerTest::test_scheduler_restart_after_stop	✅	9ms
tests/explorer/scheduler_test.py::SchedulerTest::test_split_tasks	✅	8ms
tests/explorer/scheduler_test.py::SchedulerTest::test_wait_all	✅	8ms
tests/explorer/scheduler_test.py::SchedulerTest::test_wait_all_timeout_with_multi_batch	✅	14ms
tests/explorer/workflow_test.py::WorkflowTest::test_gsm8k_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_boxed_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_complex_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_fraction_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_rm_gallery_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_workflow_resettable	✅	1ms
tests/trainer/trainer_test.py::BaseTrainerCase::test_trainer	✅	1ms
tests/trainer/trainer_test.py::TestTrainerCountdown::test_trainer	✅	301ms
tests/trainer/trainer_test.py::TestStepAheadAsyncRL::test_trainer	✅	94ms
tests/trainer/trainer_test.py::TestTrainerGSM8K::test_trainer	✅	71ms
tests/trainer/trainer_test.py::TestTrainerSFTWarmupGSM8K::test_trainer	✅	85ms
tests/trainer/trainer_test.py::TestTrainerDPO::test_trainer	✅	47ms
tests/trainer/trainer_test.py::TestFullyAsyncMode::test_fully_async_mode_0_queue	✅	104ms
tests/trainer/trainer_test.py::TestFullyAsyncMode::test_fully_async_mode_1_priority_queue	✅	104ms
tests/utils/plugin_test.py::TestPluginLoader::test_load_plugins	✅	5ms

Github Test Reporter by CTRF 💚

pan-x-c · 2025-07-17T09:02:04Z

/run-unittest

github-actions · 2025-07-17T09:25:08Z

Summary

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Other ❓	Flaky 🍂	Duration ⏱️
51	51	0	0	0	0	1.3s

Tests

Test Name	Status	Duration
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_dpo_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_mix_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_opmd_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_ppo_policy_loss	✅	1ms
tests/algorithm/policy_loss_test.py::VerlPolicyLossTest::test_sft_policy_loss	✅	1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_buffer	✅	3ms
tests/buffer/file_test.py::TestFileBuffer::test_file_reader	✅	1ms
tests/buffer/file_test.py::TestFileBuffer::test_file_writer	✅	2ms
tests/buffer/queue_test.py::TestQueueBuffer::test_priority_queue_buffer_reuse	✅	7ms
tests/buffer/queue_test.py::TestQueueBuffer::test_priority_queue_capacity	✅	2ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer_0_queue	✅	4ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer_1_priority_queue	✅	4ms
tests/buffer/queue_test.py::TestQueueBuffer::test_queue_buffer_capacity	✅	4ms
tests/buffer/sql_test.py::TestSQLBuffer::test_create_sql_buffer	✅	4ms
tests/common/config_test.py::TestConfig::test_all_examples_are_valid	✅	1ms
tests/common/config_test.py::TestConfig::test_load_default_config	✅	5ms
tests/common/experience_test.py::TestExperienceConversion::test_batch_conversion	✅	1ms
tests/common/experience_test.py::TestExperienceConversion::test_experience_model_experience_conversion	✅	1ms
tests/common/vllm_test.py::ModelWrapperTest_0::test_generate	✅	43ms
tests/common/vllm_test.py::ModelWrapperTest_1::test_generate	✅	55ms
tests/common/vllm_test.py::ModelWrapperTest_2::test_generate	✅	55ms
tests/common/vllm_test.py::ModelWrapperTest_3::test_generate	✅	42ms
tests/common/vllm_test.py::ModelWrapperTest_4::test_generate	✅	54ms
tests/common/vllm_test.py::TestAPIServer::test_api	✅	32ms
tests/common/vllm_test.py::TestTokenizer::test_assistant_token_mask	✅	1ms
tests/explorer/explorer_test.py::BaseExplorerCase::test_explorer	✅	1ms
tests/explorer/explorer_test.py::TestExplorerCountdownEval::test_explorer	✅	91ms
tests/explorer/explorer_test.py::TestExplorerCountdownNoEval::test_explorer	✅	105ms
tests/explorer/scheduler_test.py::SchedulerTest::test_concurrent_operations	✅	5ms
tests/explorer/scheduler_test.py::SchedulerTest::test_get_results	✅	21ms
tests/explorer/scheduler_test.py::SchedulerTest::test_scheduler_all_methods	✅	15ms
tests/explorer/scheduler_test.py::SchedulerTest::test_scheduler_restart_after_stop	✅	9ms
tests/explorer/scheduler_test.py::SchedulerTest::test_split_tasks	✅	8ms
tests/explorer/scheduler_test.py::SchedulerTest::test_wait_all	✅	8ms
tests/explorer/scheduler_test.py::SchedulerTest::test_wait_all_timeout_with_multi_batch	✅	14ms
tests/explorer/workflow_test.py::WorkflowTest::test_gsm8k_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_boxed_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_complex_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_fraction_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_rm_gallery_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_workflow_resettable	✅	1ms
tests/trainer/trainer_test.py::BaseTrainerCase::test_trainer	✅	1ms
tests/trainer/trainer_test.py::TestTrainerCountdown::test_trainer	✅	236ms
tests/trainer/trainer_test.py::TestStepAheadAsyncRL::test_trainer	✅	98ms
tests/trainer/trainer_test.py::TestTrainerGSM8K::test_trainer	✅	60ms
tests/trainer/trainer_test.py::TestTrainerSFTWarmupGSM8K::test_trainer	✅	85ms
tests/trainer/trainer_test.py::TestTrainerDPO::test_trainer	✅	44ms
tests/trainer/trainer_test.py::TestFullyAsyncMode::test_fully_async_mode_0_queue	✅	103ms
tests/trainer/trainer_test.py::TestFullyAsyncMode::test_fully_async_mode_1_priority_queue	✅	95ms
tests/utils/plugin_test.py::TestPluginLoader::test_load_plugins	✅	5ms

Github Test Reporter by CTRF 💚

pan-x-c added 7 commits July 15, 2025 11:46

use async model by default

c726efb

get model name from openai api

4401149

record history in model wrapper

009bba5

add tests for enable history

e4ea9f8

Merge branch 'main' into feature/model_wrapper

b73fcc4

patch vllm to get prompt id in openai api

d27977f

record history in chat completion

73904de

pan-x-c changed the title ~~[WIP] ModelWrapper automatically record Experience~~ ModelWrapper automatically record Experience Jul 16, 2025

pan-x-c added 5 commits July 17, 2025 14:52

add more tests

5c6c400

add more tests

73f3826

fix doc

6d254e3

fix tests

ff9fcbe

fix openai api

167c388

simplify config

a90fe95

chenyushuo approved these changes Jul 17, 2025

View reviewed changes

chenyushuo merged commit 00f3b27 into modelscope:main Jul 17, 2025
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

ModelWrapper automatically record Experience #123

ModelWrapper automatically record Experience #123

Uh oh!

pan-x-c commented Jul 15, 2025 •

edited

Loading

Uh oh!

pan-x-c commented Jul 16, 2025

Uh oh!

github-actions bot commented Jul 16, 2025

Uh oh!

pan-x-c commented Jul 17, 2025

Uh oh!

github-actions bot commented Jul 17, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ModelWrapper automatically record Experience #123

ModelWrapper automatically record Experience #123

Uh oh!

Conversation

pan-x-c commented Jul 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Uh oh!

pan-x-c commented Jul 16, 2025

Uh oh!

github-actions bot commented Jul 16, 2025

Summary

Tests

Uh oh!

pan-x-c commented Jul 17, 2025

Uh oh!

github-actions bot commented Jul 17, 2025

Summary

Tests

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

pan-x-c commented Jul 15, 2025 •

edited

Loading