fix: Extend add_tensor_model so that ModelDeploymentCard can be correctly picked up

[zhongdaor-nv]

Overview:

Extend add_tensor_model so that ModelDeploymentCard can be correctly picked up

Details:

Where should the reviewer start?

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

• closes GitHub issue: #xxx

Summary by CodeRabbit

• New Features

  • Added support for optional runtime configuration when deploying tensor models, enabling more granular control over deployment settings.

• Bug Fixes

  • Improved model removal process to ensure complete cleanup of associated deployment metadata during tensor model deletion.

[coderabbitai]

Walkthrough

The KServe gRPC Rust bindings are extended to support optional runtime configuration for tensor models. The add_tensor_model method now accepts an optional ModelRuntimeConfig parameter to conditionally create and persist model deployment metadata via a ModelDeploymentCard. The remove_tensor_model method adds best-effort cleanup of associated model cards.

Changes

Cohort / File(s)	Summary
Imports and types lib/bindings/python/rust/kserve_grpc.rs	Introduces llm_rs alias and imports ModelDeploymentCard, ModelInput, ModelType for model deployment metadata, and ModelRuntimeConfig from llm::local_model.
API signature updates lib/bindings/python/rust/kserve_grpc.rs	add_tensor_model extended to accept optional runtime_config: Option<ModelRuntimeConfig> parameter; pyo3 signature annotation updated to (model, checksum, engine, runtime_config=None) for Python binding.
Conditional persistence logic lib/bindings/python/rust/kserve_grpc.rs	add_tensor_model now conditionally creates a ModelDeploymentCard with Tensor-based type when runtime_config is provided, assigns the configuration, and persists via model_manager; errors propagated to Python via to_pyerr.
Cleanup enhancement lib/bindings/python/rust/kserve_grpc.rs	remove_tensor_model now attempts to remove associated model card (best-effort, silently ignores if absent) and returns Ok(()) after cleanup.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

• Verify conditional logic in add_tensor_model correctly creates and persists ModelDeploymentCard only when runtime_config is Some
• Confirm error handling and propagation to Python layer via to_pyerr is appropriate for card persistence failures
• Validate that remove_tensor_model cleanup gracefully handles missing model cards without affecting primary removal logic
• Check that new imports and type usage (ModelDeploymentCard, ModelType, ModelInput) align with broader codebase patterns

Poem

🐰 A config now flows where tensors play,
Deployment cards filed away each day,
When models depart, we clean with care,
Optional settings floating through air! ✨

Pre-merge checks

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The PR description is largely incomplete with most template sections either empty or containing only placeholder text (e.g., '#xxx' for issue number, no file pointers or change details).	Fill in the Details and Where should the reviewer start sections with specifics, complete the issue number reference, and describe the actual changes made.
Docstring Coverage	⚠️ Warning	Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%.	You can run @coderabbitai generate docstrings to improve docstring coverage.

✅ Passed checks (1 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately describes the main change: extending add_tensor_model to support ModelDeploymentCard pickup, which is the core modification evident in the changeset.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[rmccorm4]

Can you add or append to an existing test so that we can verify the ModelConfig endpoint works when runtime_config is provided?

[KrishnanPrash]

In dynamo, when a non-tensor model is unloaded, the MDC persists in etcd. From my understanding, with these changes, tensor model MDCs are explicitly removed from etcd. Does this create inconsistent cleanup behavior between model types?

[rmccorm4]

In dynamo, when a non-tensor model is unloaded, the MDC persists in etcd.

@grahamking changed this behavior in the past month or two, now MDCs are associated with model instances and should get cleaned up when the instance goes away.