Overview

This document describes the required changes to the Python CLI and SDK to support input arguments override when running a pipeline via PipelineVersionRun. This enables users to dynamically override orchestration-specific input parameters (e.g., Argo Workflow arguments) for each run, supporting use-cases such as prompt injection for Agentic AI without requiring new PipelineVersions.

API Feature Summary (Reference Only; Already Implemented)

• Protobuf schema has been updated in the backend to support:
  • input_args_override field in PipelineVersionRun (accepts OrchestrationArgsOverride)
  • orchestration_spec field in PipelineVersionRun for the merged, final spec
  • Support for Argo parameter overrides via ArgoArgsOverride and ArgoParameterOverride
• CLI/SDK should expect these fields/messages in the API responses and requests.

Semantics

• If no override is provided, values from the PipelineVersion's orchestration spec are used.
• For each run, matching parameter names are replaced by those in input_args_override.
• Unknown parameters are rejected with clear error messaging.
• All values are treated as strings, per Argo convention.

CLI Changes

1. Command Updates

• Update clarifai pipeline run to accept input argument overrides for each run.

2. Input Override Methods

• Inline via --set:

  clarifai pipeline run \
    --compute_cluster_id=... \
    --nodepool_id=... \
    --set prompt="Summarize this research paper" \
    --set temperature="0.7"
  

• File-based via --overrides-file:

  clarifai pipeline run \
    --compute_cluster_id=... \
    --nodepool_id=... \
    --overrides-file overrides.json
  

  overrides.json:

  {
    "prompt": "Summarize this research paper",
    "temperature": "0.7"
  }

• Support both inline and file-based overrides for flexibility.

3. Output Formatting

• Update output to display the final merged orchestration spec (orchestration_spec) for the run.
• Add JSON output option for programmatic inspection.

SDK Changes

1. Proto/Model Updates

• Regenerate Python proto files from the updated .proto definitions to include all relevant messages and fields:
  • OrchestrationArgsOverride
  • ArgoArgsOverride
  • ArgoParameterOverride
  • updated PipelineVersionRun with new fields

2. Client Method Updates

• Update Pipeline.run() (or equivalent) to accept an input_args_override argument.
• Validate that provided overrides match parameters defined in the PipelineVersion spec.
• Merge overrides with PipelineVersion parameters (as described in Semantics).
• Add type hints and docstrings for new fields.
• Ensure PipelineVersionRun responses expose the merged orchestration_spec.

3. Helper Methods (Optional)

• Add convenience helpers for building and validating overrides:
  • build_arg_override(parameters: dict[str, str]) -> OrchestrationArgsOverride
  • validate_arg_override(override: OrchestrationArgsOverride, pipeline_version: PipelineVersion) -> bool

4. Example SDK Usage

from clarifai_grpc.grpc.api import resources_pb2
from clarifai.client import Pipeline

pipeline = Pipeline(...existing fields...)
overrides = resources_pb2.OrchestrationArgsOverride(
    argo_args_override=resources_pb2.ArgoArgsOverride(
        parameters=[
            resources_pb2.ArgoParameterOverride(name="prompt", value="Summarize this research paper"),
            resources_pb2.ArgoParameterOverride(name="temperature", value="0.7"),
        ]
    )
)
run = pipeline.run(
    ... existing params ...
    input_args_override=overrides
)

Backward Compatibility

• ✅ Fully Backward Compatible: Existing pipeline run commands without overrides will continue to work.
• ✅ Optional Field: input_args_override is optional.
• ✅ No Breaking Changes: No existing fields are modified or removed.

Testing Requirements

1. Unit Tests

• Test pipeline run with input argument overrides (both inline and file).
• Test validation and error handling for unknown parameters.
• Test backward compatibility (run with no overrides).

2. Integration Tests

• End-to-end tests: pipeline creation, run with overrides, inspect resulting workflow spec.
• CLI command testing for all override modes.
• Error scenarios (invalid override keys, malformed file).

3. Example Code & Documentation

• Add input argument override usage examples to docs.
• Update pipeline run examples to show both inline and file-based override usage.

Security & Traceability

• Traceability: Store the final orchestration spec in PipelineVersionRun for accurate auditing and debugging.
• Validation: Enforce parameter name validation to prevent accidental misconfiguration.

Implementation Guidance

• Proto Changes: Regenerate and update Python proto files and dependent classes.
• Client & CLI Consistency: Ensure CLI and SDK methods for argument overrides follow the same validation logic and merge semantics.
• Tests and Docs: Provide comprehensive tests and update documentation to reflect override support in both inline and file-based modes.

References

• Backend proto changes: PipelineVersionRun, OrchestrationArgsOverride, ArgoArgsOverride, ArgoParameterOverride
• Existing SDK and CLI pipeline run implementations