Skip to content

Prevent pipe sink task id from logging secrets#17981

Merged
jt2594838 merged 2 commits into
apache:masterfrom
luoluoyuyu:pipe-sink-display-task-id
Jun 18, 2026
Merged

Prevent pipe sink task id from logging secrets#17981
jt2594838 merged 2 commits into
apache:masterfrom
luoluoyuyu:pipe-sink-display-task-id

Conversation

@luoluoyuyu

Copy link
Copy Markdown
Member

Description

As the title said


This PR has:

  • been self-reviewed.
    • concurrent read
    • concurrent write
    • concurrent read and write
  • added documentation for new or modified features or behaviors.
  • added Javadocs for most classes and all non-trivial methods.
  • added or updated version, license, or notice information
  • added comments explaining the "why" and the intent of the code wherever would not be obvious
    for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold
    for code coverage.
  • added integration tests.
  • been tested in a test IoTDB cluster.

Key changed/added classes (or packages if there are too many classes) in this PR

@luoluoyuyu luoluoyuyu force-pushed the pipe-sink-display-task-id branch from e6bedbc to 8225413 Compare June 18, 2026 03:39
@Caideyipi

Copy link
Copy Markdown
Collaborator

I found two issues that still look worth fixing:

  1. PipeSinkSubtaskManager now restores the raw attributeSortedString into the internal taskID, which is needed for compatibility, but the same raw string is also passed into PipeSinkSubtask and later returned by getAttributeSortedString(). That value is used as the metrics Tag.NAME in PipeDataRegionSinkMetrics and PipeSchemaRegionSinkMetrics, so secrets such as sink.scp.password=... or sink.ssl.trust-store-pwd=... can still be exposed through metric labels. Please keep the raw id only for internal map/executor keys, and use the display/sanitized attribute string for metric labels.

  2. PipeSubtaskExecutor#getSafeSubtaskStr is still used on the SUBTASK_NOT_REGISTERED fallback paths and only masks password=.... With the raw sink taskID, this can still log sensitive keys covered by PipeParameters.ValueHider, for example ssl.trust-store-pwd. It would be better to reuse the same hiding rule as ValueHider, or avoid logging raw ids in those fallback paths.

@jt2594838 jt2594838 merged commit d6d5755 into apache:master Jun 18, 2026
40 checks passed
@luoluoyuyu luoluoyuyu deleted the pipe-sink-display-task-id branch June 22, 2026 08:51
MileaRobertStefan pushed a commit to MileaRobertStefan/iotdb that referenced this pull request Jun 26, 2026
* Prevent pipe sink task id from logging secrets

* Fix pipe sink subtask compatibility
luoluoyuyu added a commit to luoluoyuyu/iotdb that referenced this pull request Jun 29, 2026
* Prevent pipe sink task id from logging secrets

* Fix pipe sink subtask compatibility
luoluoyuyu added a commit to luoluoyuyu/iotdb that referenced this pull request Jun 29, 2026
* Prevent pipe sink task id from logging secrets

* Fix pipe sink subtask compatibility
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants