Using auxiliary models in experience pipeline & OpenAI API supports stream mode#513
Merged
pan-x-c merged 12 commits intoagentscope-ai:mainfrom Mar 3, 2026
Merged
Conversation
Contributor
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
There was a problem hiding this comment.
Pull request overview
This PR wires auxiliary (judge) models into the experience processing pipeline by making operators async-capable and providing them access to auxiliary model OpenAI clients, while also refactoring ModelWrapper to infer engine type from the underlying model actor.
Changes:
- Add an async operator interface (
ExperienceOperatorV1) and updateExperiencePipelineto prepare operators asynchronously andawaitoperator processing/cleanup. - Add auxiliary model wrapper discovery (
get_auxiliary_model_wrappers) and pass auxiliary model OpenAI clients into experience operators. - Make
ModelWrapperfetch engine type from the model actor (get_engine_type) and remove config-passedengine_type.
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| trinity/explorer/workflow_runner.py | Stop passing engine_type into ModelWrapper; rely on model-reported engine type. |
| trinity/explorer/explorer.py | Reorder preparation so models are prepared before the experience pipeline; add node-affinity comment for the pipeline actor. |
| trinity/common/models/vllm_model.py | Implement get_engine_type() for vLLM-backed inference models. |
| trinity/common/models/tinker_model.py | Implement get_engine_type() for Tinker-backed inference models. |
| trinity/common/models/model.py | Add InferenceModel.get_engine_type() abstractmethod and fetch it in ModelWrapper.prepare(). |
| trinity/common/models/init.py | Rename auxiliary actor names to optionally include config name; add get_auxiliary_model_wrappers() helper. |
| trinity/common/config.py | Make DataProcessorConfig.experience_pipeline non-optional with a default factory. |
| trinity/buffer/pipelines/experience_pipeline.py | Defer operator creation to prepare(), inject auxiliary model clients, and make operator execution/close async. |
| trinity/buffer/operators/experience_operator.py | Introduce ExperienceOperatorV1 async interface + wrapper for legacy operators; add create_operators() helper. |
| trinity/buffer/operators/init.py | Export the new operator interface and factory. |
| tests/explorer/workflow_test.py | Update ModelWrapper construction after removing engine_type parameter. |
| tests/explorer/scheduler_test.py | Update dummy inference models to implement get_engine_type(). |
| tests/explorer/explorer_test.py | Add a test operator that uses auxiliary models via OpenAI async clients; configure an auxiliary model name. |
| tests/common/vllm_test.py | Update ModelWrapper construction after removing engine_type parameter. |
| tests/buffer/reward_shaping_mapper_test.py | Switch to async test and use create_operators() with await op.process(...). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
pan-x-c
changed the title
Collaborator
Author
|
/unittest-diff |
Summary
Skipped
Tests
Github Test Reporter by CTRF 💚 |
Collaborator
Author
|
/unittest-module-trainer |
Summary
Skipped
Tests
Github Test Reporter by CTRF 💚 |
Collaborator
Author
|
/gemini review |
Contributor
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
Collaborator
Author
|
/unittest-module-trainer |
Collaborator
|
/unittest-module-trainer |
Summary
Skipped
Tests
Github Test Reporter by CTRF 💚 |
Collaborator
Author
|
/unittest-module-common |
Summary
Skipped
Tests
Github Test Reporter by CTRF 💚 |
chenyushuo
approved these changes
Mar 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
As the title says
Checklist
Please check the following items before code is ready to be reviewed.