Skip to content

fix: Redact credentials in download task logs#1818

Merged
gaius-qi merged 7 commits intodragonflyoss:mainfrom
YQ-Wang:fix/redact-sensitive-creds-in-download-task-logs
May 9, 2026
Merged

fix: Redact credentials in download task logs#1818
gaius-qi merged 7 commits intodragonflyoss:mainfrom
YQ-Wang:fix/redact-sensitive-creds-in-download-task-logs

Conversation

@YQ-Wang
Copy link
Copy Markdown
Contributor

@YQ-Wang YQ-Wang commented Apr 29, 2026

Description

This change adds a debug-safe wrapper for Download messages logged by the dfdaemon gRPC download handlers.

DfdaemonDownload::download_task and DfdaemonUpload::download_task previously logged the full Download payload with {:?} at info level. That payload can include caller-provided credentials in object storage, HDFS, HuggingFace, ModelScope, and request-header fields.

The new RedactedDownload wrapper preserves the existing log shape while replacing sensitive values with ***REDACTED***.

Redacted fields include:

  • object_storage.access_key_secret
  • object_storage.session_token
  • object_storage.security_token
  • object_storage.credential_path
  • hdfs.delegation_token
  • hugging_face.token
  • model_scope.token
  • Sensitive request headers such as authorization, proxy-authorization, cookie, x-amz-security-token, x-oss-security-token, and API-key style headers

Unit tests cover structured credential redaction, request-header redaction, preservation of non-secret fields, and the final rendered Debug output.

Motivation and Context

When dfget passes object storage credentials to dfdaemon, dfdaemon currently logs the full Download message at info level. Since the generated protobuf Debug implementation prints all fields verbatim, secret access keys and session tokens can be written to dfdaemon logs.

This change keeps the operational value of the existing log message while preventing plaintext credentials from being persisted.

Test Plan

cargo test -p dragonfly-client grpc::debug

@YQ-Wang YQ-Wang requested review from a team as code owners April 29, 2026 06:37
@YQ-Wang YQ-Wang changed the title redact credentials in download task info! logs Redact credentials in download task logs Apr 29, 2026
@YQ-Wang YQ-Wang changed the title Redact credentials in download task logs fix: Redact credentials in download task logs Apr 29, 2026
@YQ-Wang YQ-Wang force-pushed the fix/redact-sensitive-creds-in-download-task-logs branch from 35090af to 2d7442e Compare April 29, 2026 06:40
Signed-off-by: Yiqing Wang <yiqingwang@roblox.com>
@YQ-Wang YQ-Wang force-pushed the fix/redact-sensitive-creds-in-download-task-logs branch from 2d7442e to 3e3ab4d Compare April 29, 2026 06:41
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 29, 2026

Codecov Report

❌ Patch coverage is 99.66216% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 44.17%. Comparing base (a3faedc) to head (1bdd0c6).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
dragonfly-client-util/src/types/redacted.rs 99.66% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1818      +/-   ##
==========================================
+ Coverage   43.59%   44.17%   +0.58%     
==========================================
  Files          92       93       +1     
  Lines       26933    27229     +296     
==========================================
+ Hits        11741    12029     +288     
- Misses      15192    15200       +8     
Files with missing lines Coverage Δ
dragonfly-client/src/grpc/dfdaemon_download.rs 4.79% <ø> (ø)
dragonfly-client/src/grpc/dfdaemon_upload.rs 0.00% <ø> (ø)
dragonfly-client-util/src/types/redacted.rs 99.66% <99.66%> (ø)

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: Yiqing Wang <yiqingwang@roblox.com>
@gaius-qi gaius-qi added the enhancement New feature or request label Apr 29, 2026
YQ-Wang and others added 5 commits April 29, 2026 01:28
Signed-off-by: Yiqing Wang <yiqingwang@roblox.com>
Signed-off-by: Gaius <gaius.qi@gmail.com>
Signed-off-by: Gaius <gaius.qi@gmail.com>
Signed-off-by: Gaius <gaius.qi@gmail.com>
@gaius-qi gaius-qi enabled auto-merge (squash) May 9, 2026 06:53
@gaius-qi gaius-qi disabled auto-merge May 9, 2026 06:53
@gaius-qi gaius-qi enabled auto-merge (squash) May 9, 2026 06:53
Copy link
Copy Markdown

@BraveY BraveY left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gaius-qi gaius-qi merged commit b29d158 into dragonflyoss:main May 9, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants