Fix openai API server setup #348

pan-x-c · 2025-10-29T07:44:18Z

Description

The current OpenAI API server uses an asynchronous startup mode. In some extreme cases, the API server may not have started properly when the explorer is running, causing the workflow runners to fail.

Checklist

Please check the following items before code is ready to be reviewed.

Code has passed all tests
Docstrings have been added/updated in Google Style
Documentation has been updated
Code is ready for review

gemini-code-assist · 2025-10-29T07:44:35Z

Summary of Changes

Hello @pan-x-c, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request resolves a race condition where the OpenAI API server might not be fully operational when the explorer attempts to utilize it, potentially causing workflow failures. The changes refactor the API server's startup mechanism to ensure that all necessary API servers are explicitly initialized and their readiness is confirmed during the explorer's preparation phase, thereby guaranteeing their availability and stability.

Highlights

Centralized API Server Startup: The asynchronous calls to run_api_server have been removed from the initial model creation phase within create_inference_models, centralizing the API server startup logic.
Explicit API Server Readiness: The explorer's prepare method now explicitly calls run_api_server for all models and awaits their completion using asyncio.gather, ensuring that all OpenAI API servers are fully initialized before the explorer proceeds with its operations.
Robust API Server Method: The run_api_server method in vllm_model.py has been enhanced to be idempotent and conditional. It now checks if the API is enabled and if the server is already running, returning early instead of raising an error, which prevents issues with multiple calls.
Synchronous Debug Startup: For debugging purposes, the create_debug_inference_model function now explicitly awaits the API server startup, guaranteeing its readiness during debug sessions.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request effectively addresses a race condition by ensuring the OpenAI API servers are started synchronously and awaited before the explorer proceeds. The changes in trinity/explorer/explorer.py and trinity/common/models/__init__.py correctly implement this synchronous startup. Additionally, making the run_api_server method in trinity/common/models/vllm_model.py idempotent is a good improvement for robustness. I have a couple of suggestions to improve code clarity and maintainability.

trinity/common/models/vllm_model.py

trinity/explorer/explorer.py

pan-x-c · 2025-10-29T08:59:56Z

/unittest-diff

github-actions · 2025-10-29T09:15:46Z

Summary

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Other ❓	Flaky 🍂	Duration ⏱️
69	68	0	1	0	0	869ms

Skipped

Tests	Status
tests/explorer/workflow_test.py::TestAgentScopeWorkflowAdapter::test_adapter	skipped ⏭️

Tests

Test Name	Status	Duration
tests/common/config_test.py::TestConfig::test_all_examples_are_valid	✅	32ms
tests/common/config_test.py::TestConfig::test_config_flatten	✅	1ms
tests/common/config_test.py::TestConfig::test_continue_from_checkpoint_is_valid	✅	1ms
tests/common/config_test.py::TestConfig::test_default_workflow	✅	1ms
tests/common/config_test.py::TestConfig::test_load_default_config	✅	3ms
tests/common/config_test.py::TestConfig::test_max_token_len_per_gpu_set_correctly	✅	1ms
tests/common/config_test.py::TestConfig::test_update_config_from_ray_cluster	✅	1ms
tests/common/experience_test.py::TestEID::test_eid_properties	✅	1ms
tests/common/experience_test.py::TestExperience::test_action_mask_and_logprobs_type	✅	1ms
tests/common/experience_test.py::TestExperience::test_assertions	✅	1ms
tests/common/experience_test.py::TestExperience::test_dpo_experience	✅	1ms
tests/common/experience_test.py::TestExperience::test_gather	✅	1ms
tests/common/experience_test.py::TestExperience::test_hf_datasets_conversion	✅	1ms
tests/common/experience_test.py::TestExperience::test_multi_turn_experience	✅	1ms
tests/common/experience_test.py::TestExperience::test_serialize_deserialize	✅	1ms
tests/common/experience_test.py::TestExperience::test_single_turn_experience	✅	1ms
tests/common/experience_test.py::TestExperience::test_to_dict	✅	1ms
tests/common/experience_test.py::TestExperienceConversion::test_batch_conversion	✅	1ms
tests/common/experience_test.py::TestExperienceConversion::test_dpo_experience_batch_conversion	✅	1ms
tests/common/experience_test.py::TestExperienceConversion::test_experience_model_experience_conversion	✅	1ms
tests/common/experience_test.py::TestExperienceConversion::test_gather_experiences_with_custom_fields	✅	1ms
tests/common/experience_test.py::TestExperienceConversion::test_multiturn_experience_batch_converstion	✅	1ms
tests/common/vllm_test.py::ModelWrapperTest_0::test_generate	✅	55ms
tests/common/vllm_test.py::ModelWrapperTest_1::test_generate	✅	34ms
tests/common/vllm_test.py::ModelWrapperTest_2::test_generate	✅	46ms
tests/common/vllm_test.py::TestModelLen_0::test_model_len	✅	20ms
tests/common/vllm_test.py::TestModelLen_1::test_model_len	✅	22ms
tests/common/vllm_test.py::TestAPIServer::test_api	✅	27ms
tests/common/vllm_test.py::TestAsyncAPIServer::test_api_async	✅	27ms
tests/common/vllm_test.py::TestTokenizer::test_action_mask	✅	1ms
tests/common/vllm_test.py::TestTokenizer::test_action_mask_with_tools	✅	1ms
tests/common/vllm_test.py::TestAPIServerToolCall_0_deepseek_r1::test_api_tool_calls	✅	22ms
tests/common/vllm_test.py::TestAPIServerToolCall_1::test_api_tool_calls	✅	22ms
tests/explorer/explorer_test.py::TestExplorerCountdownEval::test_explorer	✅	63ms
tests/explorer/explorer_test.py::TestExplorerGSM8KRULERNoEval::test_explorer	✅	58ms
tests/explorer/explorer_test.py::TestExplorerGSM8k::test_explorer	✅	204ms
tests/explorer/explorer_test.py::ServeTest::test_serve	✅	69ms
tests/explorer/scheduler_test.py::SchedulerTest::test_async_workflow	✅	5ms
tests/explorer/scheduler_test.py::SchedulerTest::test_concurrent_operations	✅	5ms
tests/explorer/scheduler_test.py::SchedulerTest::test_get_results	✅	23ms
tests/explorer/scheduler_test.py::SchedulerTest::test_multi_step_execution	✅	5ms
tests/explorer/scheduler_test.py::SchedulerTest::test_non_repeatable_workflow	✅	5ms
tests/explorer/scheduler_test.py::SchedulerTest::test_scheduler_all_methods	✅	15ms
tests/explorer/scheduler_test.py::SchedulerTest::test_scheduler_restart_after_stop	✅	10ms
tests/explorer/scheduler_test.py::SchedulerTest::test_split_tasks	✅	8ms
tests/explorer/scheduler_test.py::SchedulerTest::test_stepwise_experience_eid	✅	5ms
tests/explorer/scheduler_test.py::SchedulerTest::test_wait_all	✅	8ms
tests/explorer/scheduler_test.py::SchedulerTest::test_wait_all_timeout_with_multi_batch	✅	14ms
tests/explorer/step_wise_workflow_test.py::WorkflowTest::test_reward_propagation_workflow_0	✅	1ms
tests/explorer/step_wise_workflow_test.py::WorkflowTest::test_reward_propagation_workflow_1	✅	1ms
tests/explorer/step_wise_workflow_test.py::WorkflowTest::test_step_wise_reward_workflow_0	✅	1ms
tests/explorer/step_wise_workflow_test.py::WorkflowTest::test_step_wise_reward_workflow_1	✅	1ms
tests/explorer/step_wise_workflow_test.py::WorkflowTest::test_workflows_raise_error	✅	1ms
tests/explorer/step_wise_workflow_test.py::WorkflowTest::test_workflows_stop_at_max_env_steps	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_gsm8k_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_boxed_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_complex_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_eval_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_fraction_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_math_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_rm_gallery_workflow	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_workflow_repeatable_0	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_workflow_repeatable_1	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_workflow_resettable_0	✅	1ms
tests/explorer/workflow_test.py::WorkflowTest::test_workflow_resettable_1	✅	1ms
tests/explorer/workflow_test.py::MultiTurnWorkflowTest_0::test_multi_turn_workflow	✅	18ms
tests/explorer/workflow_test.py::MultiTurnWorkflowTest_1::test_multi_turn_workflow	✅	19ms
tests/explorer/workflow_test.py::TestAgentScopeWorkflowAdapter::test_adapter	⏭️	1ms
tests/explorer/workflow_test.py::TestWorkflowRunner::test_workflow_runner	✅	1ms

Github Test Reporter by CTRF 💚

trinity/common/config.py

garyzhang99

LGTM

fix api server start

d535a6c

gemini-code-assist bot reviewed Oct 29, 2025

View reviewed changes

trinity/common/models/vllm_model.py Outdated Show resolved Hide resolved

trinity/explorer/explorer.py Outdated Show resolved Hide resolved

pan-x-c added 2 commits October 29, 2025 16:23

simplify openai server setup

150ed00

add auxiliary model tests

92b3784

garyzhang99 reviewed Oct 29, 2025

View reviewed changes

trinity/common/config.py Show resolved Hide resolved

fix config

7b9659f

garyzhang99 approved these changes Oct 29, 2025

View reviewed changes

pan-x-c merged commit 9efaec8 into agentscope-ai:main Oct 29, 2025
1 check passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix openai API server setup #348

Fix openai API server setup #348

Uh oh!

pan-x-c commented Oct 29, 2025 •

edited

Loading

Uh oh!

gemini-code-assist bot commented Oct 29, 2025

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

Uh oh!

Uh oh!

pan-x-c commented Oct 29, 2025

Uh oh!

github-actions bot commented Oct 29, 2025

Uh oh!

Uh oh!

garyzhang99 left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Fix openai API server setup #348

Fix openai API server setup #348

Uh oh!

Conversation

pan-x-c commented Oct 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Uh oh!

gemini-code-assist bot commented Oct 29, 2025

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

pan-x-c commented Oct 29, 2025

Uh oh!

github-actions bot commented Oct 29, 2025

Summary

Skipped

Tests

Uh oh!

Uh oh!

garyzhang99 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

pan-x-c commented Oct 29, 2025 •

edited

Loading