Skip to content

[Benchmarks] Print all important env vars in debug logs #20415

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: sycl
Choose a base branch
from
+14 −8
Open

[Benchmarks] Print all important env vars in debug logs #20415

Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1c171cb
[Benchmarks] Print all important env vars in debug logs
PatKamin Oct 21, 2025
d24f622
Print only added paths to PATH and LD_LIBRARY_PATH
PatKamin Oct 22, 2025
aa3f3ea
Replace env vars where appropriate
PatKamin Oct 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 15 additions & 7 deletions devops/scripts/benchmarks/utils/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,28 +52,36 @@ def run(

for ldlib in ld_library:
if os.path.isdir(ldlib):
env["LD_LIBRARY_PATH"] = (
ldlib + os.pathsep + env.get("LD_LIBRARY_PATH", "")
env_vars["LD_LIBRARY_PATH"] = os.pathsep.join(
filter(None, [ldlib, env_vars.get("LD_LIBRARY_PATH", "")])
)
else:
log.warning(f"LD_LIBRARY_PATH component does not exist: {ldlib}")

# order is important, we want provided sycl rt libraries to be first
if add_sycl:
sycl_bin_path = os.path.join(options.sycl, "bin")
env["PATH"] = sycl_bin_path + os.pathsep + env.get("PATH", "")
env_vars["PATH"] = os.pathsep.join(
filter(None, [sycl_bin_path, env_vars.get("PATH", "")])
)
sycl_lib_path = os.path.join(options.sycl, "lib")
env["LD_LIBRARY_PATH"] = (
sycl_lib_path + os.pathsep + env.get("LD_LIBRARY_PATH", "")
env_vars["LD_LIBRARY_PATH"] = os.pathsep.join(
filter(None, [sycl_lib_path, env_vars.get("LD_LIBRARY_PATH", "")])
)

env.update(env_vars)

command_str = " ".join(command)
env_str = " ".join(f"{key}={value}" for key, value in env_vars.items())
full_command_str = f"{env_str} {command_str}".strip()
log.debug(f"Running: {full_command_str}")

# Prepend new value to existing env value
for key, value in env_vars.items():
old_value = env.get(key, "")
if old_value:
env[key] = os.pathsep.join([value, old_value])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if key on env_vars is not a path, but something existing in env which user wanted to overwrite?
This code is getting more and more complex...

Maybe we can just assume that neither PATH, nor LD_LIBRARY_PATH is not present in env_vars (assert on this) and assume that env_vars overwrite vars and append only PATHS and LD_LIBRARY_PATH from ld_library and add_sycl params.

Sorry that I did not realized this earlier.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this has to be handled well. Unluckily, this assumption is not true, I've checked that. Proposed another solution where there is a neat check and paths are added only to PATH and LD_LIBRARY_PATH. These are the only env vars used in the framework that are lists of paths. All other are just replaced.

else:
env[key] = value

# Normalize input to bytes if it's a str
if isinstance(input, str):
input_bytes = input.encode()
Expand Down
Loading