Skip to content

Add NodeMetadata.Config property bag and expose ClusterAssignment in LaunchPlan#6173

Closed
EngHabu wants to merge 1 commit intomasterfrom
union/upstream-408f79d941d364152c58145313d3f313ce6213f1
Closed

Add NodeMetadata.Config property bag and expose ClusterAssignment in LaunchPlan#6173
EngHabu wants to merge 1 commit intomasterfrom
union/upstream-408f79d941d364152c58145313d3f313ce6213f1

Conversation

@EngHabu
Copy link
Contributor

@EngHabu EngHabu commented Jan 15, 2025

Why are the changes needed?

This allows plugins to pass execution instructions to the backend plugin without having to introduce new protos..

What changes were proposed in this pull request?

Allow passing cluster_assignment in Workflow NodeMetadata and LaunchPlan.

How was this patch tested?

Deployed to an internal tenant and verified the bits are passed correctly.

Setup process

Screenshots

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Summary by Bito

This PR introduces a new config property bag in NodeMetadata enabling plugins to pass execution instructions without new proto definitions, and adds support for cluster_assignment in LaunchPlan specifications. The changes span across multiple language implementations including Go, TypeScript, Python, and Rust, enhancing the flexibility of plugin configuration and execution control.

Unit tests added: False

Estimated effort to review (1-5, lower is better): 5

…LaunchPlan

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>
@flyte-bot
Copy link
Collaborator

Code Review Agent Run Status

  • Limitations and other issues: ❌ Failure - The AI Code Review Agent skipped reviewing this change because it is configured to exclude certain pull requests based on the source/target branch or the pull request status. You can change the settings here, or contact the agent instance creator at eduardo@union.ai.

@codecov
Copy link

codecov bot commented Jan 15, 2025

Codecov Report

Attention: Patch coverage is 16.66667% with 10 lines in your changes missing coverage. Please review.

Project coverage is 33.25%. Comparing base (e0833b7) to head (bf93770).
Report is 113 commits behind head on master.

Files with missing lines Patch % Lines
...lyteidl/gen/pb-go/flyteidl/admin/launch_plan.pb.go 16.66% 5 Missing ⚠️
flyteidl/gen/pb-go/flyteidl/core/workflow.pb.go 16.66% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6173      +/-   ##
==========================================
- Coverage   37.05%   33.25%   -3.81%     
==========================================
  Files        1318     1020     -298     
  Lines      132627   108679   -23948     
==========================================
- Hits        49151    36138   -13013     
+ Misses      79226    69344    -9882     
+ Partials     4250     3197    -1053     
Flag Coverage Δ
unittests-datacatalog 51.58% <ø> (ø)
unittests-flyteadmin ?
unittests-flytecopilot 30.99% <ø> (ø)
unittests-flytectl 62.29% <ø> (ø)
unittests-flyteidl 7.23% <16.66%> (+<0.01%) ⬆️
unittests-flyteplugins 53.85% <ø> (ø)
unittests-flytepropeller 42.69% <ø> (ø)
unittests-flytestdlib 55.27% <ø> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@EngHabu EngHabu closed this Jan 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants