upgrade zero inflated lognormal loss, support export structure path #544
Conversation
* Updated Docker image
yangxudong
changed the title
…upgrade pre-commit hooks
…upgrade pre-commit hooks
…upgrade pre-commit hooks
…upgrade pre-commit hooks
…upgrade pre-commit hooks
…upgrade pre-commit hooks
…upgrade pre-commit hooks
…upgrade pre-commit hooks
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
easy_rec/python/utils/test_utils.py
Outdated
| @@ -100,8 +98,7 @@ | |||
| cmd_str = cmd_str.replace('\r', ' ').replace('\n', ' ') | |||
| logging.info('RUNCMD: %s > %s 2>&1 ' % (cmd_str, log_file)) | |||
| with open(log_file, 'w') as lfile: | |||
| proc = subprocess.Popen( | |||
| cmd_str, stdout=lfile, stderr=subprocess.STDOUT, shell=True, env=env) | |||
| proc = subprocess.Popen(cmd_str, stdout=lfile, stderr=subprocess.STDOUT, shell=True, env=env) | |||
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information High test
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 2 months ago
The issue should be fixed by ensuring that sensitive data, such as passwords present in environment variables (e.g., redis_passwd), is not included in log output.
To fix this, the run_cmd function in easy_rec/python/utils/test_utils.py (lines 100-111) should be modified so that command strings are either (1) not logged at all or (2) masked/redacted to replace any obvious sensitive values (e.g., --redis_passwd xxx) with a constant marker (e.g., --redis_passwd ******) before logging. A generic safe approach is to redact the value of --redis_passwd (and potentially other future flags like --password). This can be implemented via regex replacement before issuing the log statement.
Only easy_rec/python/utils/test_utils.py needs modification: add masking logic before the log statement in run_cmd.
| @@ -100,7 +100,14 @@ | ||
| def run_cmd(cmd_str, log_file, env=None): | ||
| """Run a shell cmd.""" | ||
| cmd_str = cmd_str.replace('\r', ' ').replace('\n', ' ') | ||
| logging.info('RUNCMD: %s > %s 2>&1 ' % (cmd_str, log_file)) | ||
| # redact any instance of --redis_passwd <value> in logs | ||
| import re | ||
| def redact_password(cmd): | ||
| # redact patterns like --redis_passwd something (allow quoted/space values) | ||
| pattern = r'(--redis_passwd\s+)([^\s"\']+|"[^"]*"|\'[^\']*\')' | ||
| return re.sub(pattern, r'\1******', cmd) | ||
| safe_cmd_str = redact_password(cmd_str) | ||
| logging.info('RUNCMD: %s > %s 2>&1 ' % (safe_cmd_str, log_file)) | ||
| with open(log_file, 'w') as lfile: | ||
| proc = subprocess.Popen( | ||
| cmd_str, stdout=lfile, stderr=subprocess.STDOUT, shell=True, env=env) |
No description provided.