-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Description
Sub-issue of #12053
Feature Area
/area backend
/area sdk
What feature would you like to see?
As a KFP component author, I want to specify a custom_path for an output artifact that resides in a PVC, so that the KFP Launcher uploads it directly from the PVC to S3 without intermediate copying. This involves adding a custom_path field to the RuntimeArtifact protobuf message and initializing it in the Python Artifact class. The Python executor also needs to pass this field to the KFP Launcher.
This doesn't directly depend on the workspace feature.
Example Protobuf:
message RuntimeArtifact {
// The local path used to upload the artifact.
optional string custom_path = 7;
}
Example custom_path property on the Artifact class:
class Artifact:
def __init__(self,
name: Optional[str] = None,
uri: Optional[str] = None,
metadata: Optional[Dict] = None) -> None:
self.custom_path: str = None
Acceptance Criteria:
RuntimeArtifact Protobuf changes are incorporated
Go types are recompiled (make -c api golang)
The output Artifact class now has a custom_path property
I can call set_path on the output artifact
After setting custom_path, the output artifact's path property returns the custom_path
The Python executor returns the custom_path in the executor output for Launcher
The Launcher consults custom_path first and then fallsback to the local path.
Love this idea? Give it a 👍.