Skip to content

[feature] Add custom_path field to RuntimeArtifact object #12386

@alyssacgoins

Description

@alyssacgoins

Sub-issue of #12053

Feature Area

/area backend
/area sdk

What feature would you like to see?

As a KFP component author, I want to specify a custom_path for an output artifact that resides in a PVC, so that the KFP Launcher uploads it directly from the PVC to S3 without intermediate copying. This involves adding a custom_path field to the RuntimeArtifact protobuf message and initializing it in the Python Artifact class. The Python executor also needs to pass this field to the KFP Launcher.

This doesn't directly depend on the workspace feature.

Example Protobuf:

message RuntimeArtifact {
  // The local path used to upload the artifact.
  optional string custom_path = 7;
}

Example custom_path property on the Artifact class:

  class Artifact:
    def __init__(self,
                 name: Optional[str] = None,
                 uri: Optional[str] = None,
                 metadata: Optional[Dict] = None) -> None:
        self.custom_path: str = None

Acceptance Criteria:

RuntimeArtifact Protobuf changes are incorporated
Go types are recompiled (make -c api golang)
The output Artifact class now has a custom_path property
I can call set_path on the output artifact
After setting custom_path, the output artifact's path property returns the custom_path
The Python executor returns the custom_path in the executor output for Launcher
The Launcher consults custom_path first and then fallsback to the local path.


Love this idea? Give it a 👍.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions