Skip to content

Fix vllm logging #35023

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
damccorm wants to merge 3 commits into from
Closed

Fix vllm logging #35023

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
8dc305d
Fix vllm logging
damccorm May 21, 2025
c0a35cd
Dont change logging in main process either
damccorm May 21, 2025
2bebac5
Strings
damccorm May 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion sdks/python/apache_beam/ml/inference/vllm_inference.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@
from openai import OpenAI

try:
os.environ['VLLM_CONFIGURE_LOGGING'] = "0"
import vllm # pylint: disable=unused-import
logging.info('vllm module successfully imported.')
except ModuleNotFoundError:
Expand Down Expand Up @@ -67,9 +68,12 @@ class OpenAIChatMessage():
def start_process(cmd) -> tuple[subprocess.Popen, int]:
port, = subprocess_server.pick_port(None)
cmd = [arg.replace('{{PORT}}', str(port)) for arg in cmd] # pylint: disable=not-an-iterable
vllm_env = os.environ.copy()
# Use default logging to avoid vllm breaking Beam's logging
vllm_env["VLLM_CONFIGURE_LOGGING"] = "0"
logging.info("Starting service with %s", str(cmd).replace("',", "'"))
process = subprocess.Popen(
cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, env=vllm_env)

# Emit the output of this command as info level logging.
def log_stdout():
Expand Down
Loading