[Improvement-17986][task-plugin] Support parameter replacement in Flink and FlinkStream task by macdoor · Pull Request #17987 · apache/dolphinscheduler

macdoor · 2026-02-21T12:30:12Z

Purpose of the pull request

Add parameter replacement support for Flink and Flink Stream task plugins. initScript and rawScript can use DolphinScheduler built-in and custom parameters (e.g. ${system.biz.date}, ${localParams.xxx}), consistent with other task types.

close #17986

Brief change log

In FlinkTask and FlinkStreamTask, apply ParameterUtils.convertParameterPlaceholders() to initScript and rawScript in init() using taskExecutionContext.getPrepareParamsMap()
Add unit tests (FlinkTaskTest, FlinkStreamTaskTest) to verify parameter replacement for ${var} placeholders and $[yyyyMMdd] time expressions
No new dependencies; reuses existing ParameterUtils and task API

Verify this pull request

This change added tests and can be verified as follows:

Added FlinkTaskTest and FlinkStreamTaskTest with test cases:
- testInitReplacesPlaceholdersInInitScriptAndRawScript(): Verifies ${var} replacement with custom parameters
- testInitReplacesTimePlaceholderWhenParamMapContainsScheduleTime(): Verifies $[yyyyMMdd] time placeholder replacement
Run tests: mvn test -pl dolphinscheduler-task-plugin/dolphinscheduler-task-flink,dolphinscheduler-task-plugin/dolphinscheduler-task-flink-stream -Dtest=FlinkTaskTest,FlinkStreamTaskTest
Manually verified: Create a Flink (or Flink Stream) task with placeholders in init script or main script (e.g. ${system.biz.date}), run the workflow and confirm the placeholders are replaced correctly at runtime

Pull Request Notice

If your pull request contains incompatible change, you should also add it to docs/docs/en/guide/upgrade/incompatible.md

closes #17986

boring-cyborg · 2026-02-21T12:30:14Z

Thanks for opening this pull request! Please check out our contributing guidelines. (https://github.com/apache/dolphinscheduler/blob/dev/docs/docs/en/contribute/join/pull-request.md)

SbloodyS

LGTM. Thanks.

sonarqubecloud · 2026-02-24T03:43:07Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
86.7% Coverage on New Code
8.7% Duplication on New Code

See analysis details on SonarQube Cloud

ruanwenjun

It's better to move

FileUtils.generateScriptFile(taskExecutionContext, flinkParameters);

at getScript method, then we can avoid use mockito at UT.

We do not recommend using Mockito in UT, using Mockito makes UT difficult to maintain.

dolphinscheduler-task-plugin/dolphinscheduler-task-flink/pom.xml

dolphinscheduler-task-plugin/dolphinscheduler-task-flink-stream/pom.xml

...link-stream/src/main/java/org/apache/dolphinscheduler/plugin/task/flink/FlinkStreamTask.java

...eduler-task-flink/src/main/java/org/apache/dolphinscheduler/plugin/task/flink/FlinkTask.java

…cript(), remove Mockito from FlinkTaskTest - Move FileUtils.generateScriptFile() from init() to getScript() - Apply ParameterUtils.convertParameterPlaceholders() in getScript() before script generation - Refactor FlinkTaskTest without Mockito, use @tempdir and assert on generated file content Ref: PR apache#17987 Co-authored-by: Cursor <cursoragent@cursor.com>

…s without Mockito - Move FileUtils.generateScriptFile() from init() to getScript() - Apply ParameterUtils.convertParameterPlaceholders() in getScript() before script generation - Refactor FlinkTaskTest and FlinkStreamTaskTest without Mockito - Use @tempdir and assert on generated file content Ref: PR apache#17987 Co-authored-by: Cursor <cursoragent@cursor.com>

…nk-stream junit-jupiter, mockito-junit-jupiter, mockito-inline are already in root pom Ref: PR apache#17987 review Co-authored-by: Cursor <cursoragent@cursor.com>

…e site - Remove param replacement from init(), apply in resolveFlinkJdbcUrl/resolveInitScriptContent/resolveMainScriptContent - Consistent with PR apache#17987 review: move replacement to where values are used Co-authored-by: Cursor <cursoragent@cursor.com>

...-stream/src/test/java/org/apache/dolphinscheduler/plugin/task/flink/FlinkStreamTaskTest.java

…subclasses Same package allows direct access to protected getScript() Ref: PR apache#17987 review Co-authored-by: Cursor <cursoragent@cursor.com>

…s without Mockito - Move FileUtils.generateScriptFile() from init() to getScript() - Apply ParameterUtils.convertParameterPlaceholders() in getScript() before script generation - Refactor FlinkTaskTest and FlinkStreamTaskTest without Mockito - Use @tempdir and assert on generated file content Ref: PR apache#17987 Co-authored-by: Cursor <cursoragent@cursor.com>

…nk-stream junit-jupiter, mockito-junit-jupiter, mockito-inline are already in root pom Ref: PR apache#17987 review Co-authored-by: Cursor <cursoragent@cursor.com>

…subclasses Same package allows direct access to protected getScript() Ref: PR apache#17987 review Co-authored-by: Cursor <cursoragent@cursor.com>

...er-task-flink/src/test/java/org/apache/dolphinscheduler/plugin/task/flink/FlinkTaskTest.java

...-stream/src/test/java/org/apache/dolphinscheduler/plugin/task/flink/FlinkStreamTaskTest.java

…ue(contains) in Flink tests - Replace assertTrue(contains) with assertEquals in FlinkTaskTest and FlinkStreamTaskTest - Remove class-level Javadoc per review Ref: PR apache#17987 Co-authored-by: Cursor <cursoragent@cursor.com>

…nk and FlinkStream task

…d unit tests for parameter replacement - Always apply parameter replacement even when paramsMap is empty (to support time placeholders like $[yyyyMMdd]) - Replace Chinese comments with English - Add FlinkTaskTest and FlinkStreamTaskTest with test cases for parameter replacement Co-authored-by: Cursor <cursoragent@cursor.com>

…nk and FlinkStream task - Replace parameter placeholders in initScript and rawScript for FlinkTask and FlinkStreamTask - Add unit tests for parameter replacement and time placeholders with empty params - Add test dependencies to task-flink and task-flink-stream modules

…s without Mockito - Move FileUtils.generateScriptFile() from init() to getScript() - Apply ParameterUtils.convertParameterPlaceholders() in getScript() before script generation - Refactor FlinkTaskTest and FlinkStreamTaskTest without Mockito - Use @tempdir and assert on generated file content Ref: PR apache#17987 Co-authored-by: Cursor <cursoragent@cursor.com>

…nk-stream junit-jupiter, mockito-junit-jupiter, mockito-inline are already in root pom Ref: PR apache#17987 review Co-authored-by: Cursor <cursoragent@cursor.com>

…subclasses Same package allows direct access to protected getScript() Ref: PR apache#17987 review Co-authored-by: Cursor <cursoragent@cursor.com>

…ue(contains) in Flink tests - Replace assertTrue(contains) with assertEquals in FlinkTaskTest and FlinkStreamTaskTest - Remove class-level Javadoc per review Ref: PR apache#17987 Co-authored-by: Cursor <cursoragent@cursor.com>

macdoor requested review from Gallardot, SbloodyS and caishunfeng as code owners February 21, 2026 12:30

github-actions bot added backend test labels Feb 21, 2026

github-actions bot assigned macdoor Feb 21, 2026

SbloodyS added first time contributor First-time contributor improvement make more easy to user or prompt friendly labels Feb 22, 2026

SbloodyS added this to the 3.4.1 milestone Feb 22, 2026

SbloodyS previously approved these changes Feb 24, 2026

View reviewed changes

SbloodyS requested a review from ruanwenjun February 24, 2026 02:50

ruanwenjun reviewed Feb 24, 2026

View reviewed changes

macdoor dismissed SbloodyS’s stale review via e245f35 February 24, 2026 11:51

macdoor force-pushed the improvement-17986 branch from 9a16038 to 4e40f8d Compare February 24, 2026 12:07

ruanwenjun reviewed Feb 25, 2026

View reviewed changes

...-stream/src/test/java/org/apache/dolphinscheduler/plugin/task/flink/FlinkStreamTaskTest.java Outdated Show resolved Hide resolved

ruanwenjun force-pushed the improvement-17986 branch from 6554359 to bfd71ea Compare February 25, 2026 03:59

ruanwenjun reviewed Feb 25, 2026

View reviewed changes

xiaoxiong.duan and others added 2 commits February 25, 2026 13:44

[Improvement-17986][task-plugin] Support parameter replacement in Fli…

cf7a5f4

…nk and FlinkStream task

xiaoxiong.duan and others added 5 commits February 25, 2026 13:44

chore(flink): remove duplicate test deps from task-flink and task-fli…

a3d03d3

…nk-stream junit-jupiter, mockito-junit-jupiter, mockito-inline are already in root pom Ref: PR apache#17987 review Co-authored-by: Cursor <cursoragent@cursor.com>

refactor(flink-test): call task.getScript() directly, remove ForTest …

0ad461f

…subclasses Same package allows direct access to protected getScript() Ref: PR apache#17987 review Co-authored-by: Cursor <cursoragent@cursor.com>

ruanwenjun force-pushed the improvement-17986 branch from 2ebdd88 to 2660988 Compare February 25, 2026 05:44

Merge branch 'dev' into improvement-17986

de2e1a4

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

[Improvement-17986][task-plugin] Support parameter replacement in Flink and FlinkStream task#17987

[Improvement-17986][task-plugin] Support parameter replacement in Flink and FlinkStream task#17987
macdoor wants to merge 8 commits intoapache:devfrom
macdoor:improvement-17986

macdoor commented Feb 21, 2026 •

edited by SbloodyS

Loading

Uh oh!

boring-cyborg bot commented Feb 21, 2026

Uh oh!

SbloodyS left a comment

Uh oh!

sonarqubecloud bot commented Feb 24, 2026

Uh oh!

ruanwenjun left a comment •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Comments

Conversation

macdoor commented Feb 21, 2026 • edited by SbloodyS Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose of the pull request

Brief change log

Verify this pull request

Pull Request Notice

Uh oh!

boring-cyborg bot commented Feb 21, 2026

Uh oh!

SbloodyS left a comment

Choose a reason for hiding this comment

Uh oh!

sonarqubecloud bot commented Feb 24, 2026

Quality Gate passed

Uh oh!

ruanwenjun left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

macdoor commented Feb 21, 2026 •

edited by SbloodyS

Loading

ruanwenjun left a comment •

edited

Loading