Skip to content

Commit 57580b8

Browse files
[ENH]: Add save_model MCP tool for persisting estimators via sktime MLflow (#48)
#### Reference Issues/PRs Fixes #31 #### What does this implement/fix? Explain your changes. Implements the requested `save_model` MCP tool for `sktime-mcp`. This PR adds a new MCP tool, `save_model`, to allow persistent storage of instantiated estimators and pipelines via `sktime.utils.mlflow_sktime.save_model`. For `docs`: updated user-facing docs and examples to document the new tool, clarify the current local filesystem path behavior of sktime's save_model API, and mention the MLflow runtime requirement Changes included: - added `src/sktime_mcp/tools/save_model.py` with `save_model_tool` . - resolved estimator handles to the underlying estimator instance through the handle manager - registered the tool in `src/sktime_mcp/server.py` with schema and dispatch wiring - added unit coverage in `tests/test_core.py` checked if run locally or not with this command `python -m pytest tests/test_core.py::TestTools::test_save_model_tool -q` <img width="1856" height="135" alt="image" src="https://github.com/user-attachments/assets/fbaf5475-8610-4be6-a09f-43fd7e0297da" /> - updated user-facing docs and examples to document the new too #### Does your contribution introduce a new dependency? If yes, which one? No but ,the tool relies on `sktime`'s MLflow integration (`sktime.utils.mlflow_sktime.save_model`), so MLflow must be available in the runtime environment for the tool to work. I documented this behavior, but did not add `mlflow` to project dependencies . #### What should reviewers focus on? I’d really appreciate feedback on a few specific areas: 1. Does `save_model` feel like the right MCP interface for handling persistence in this project? 2. Should we be more explicit in the contract and say “local filesystem path” instead of the broader “path or URI”? 3. Does it make sense to keep MLflow as a documented/runtime requirement, or should it be added as an explicit dependency #### Any other comments? The implementation follows the documented `sktime.utils.mlflow_sktime.save_model` API, which describes saving to a local path on the filesystem. I aligned the documentation with that behavior to avoid overstating URI support. #### PR checklist ##### For all contributions - [ ] I've added myself to the `https://github.com/alan-turing-institute/sktime/blob/main/.all-contributorsrc` . - [ ] Optionally, I've updated sktime's `https://github.com/alan-turing-institute/sktime/blob/main/CODEOWNERS` to receive notifications about future changes to these files. - [x] I've added unit tests and made sure they pass locally. ##### For new estimators - [ ] Not applicable, this PR does not add a new estimator. - [ ] Not applicable, this PR does not add or modify estimator example notebooks. ran the test locally they are pssing <img width="1840" height="176" alt="image" src="https://github.com/user-attachments/assets/2013fff5-4792-48e3-9ec4-9958aa16ed81" />
1 parent f6765fc commit 57580b8

7 files changed

Lines changed: 216 additions & 8 deletions

File tree

README.md

Lines changed: 29 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ This MCP is **not** just documentation or static code analysis. It is a **semant
2222

2323
2. **Registry-First** - Instead of `File → Class → Infer Relationships`, we do `Registry → Semantics → Safe Execution`.
2424

25-
3. **Minimal MCP Surface** - Exposes only what an LLM needs: Discovery, Description, Instantiation, Execution.
25+
3. **Minimal MCP Surface** - Exposes only what an LLM needs: Discovery, Description, Instantiation, Execution, and model persistence.
2626

2727
## 🛠️ Installation
2828

@@ -235,9 +235,34 @@ Execute a complete workflow: load dataset, fit estimator, and generate predictio
235235

236236
---
237237

238+
#### 9. `save_model`
239+
Persist a fitted estimator or pipeline handle to a local filesystem path using `sktime.utils.mlflow_sktime.save_model`.
240+
241+
**Arguments:**
242+
- `estimator_handle` (required): Handle from `instantiate_estimator` or `instantiate_pipeline`
243+
- `path` (required): Local filesystem path where the model should be saved
244+
- `mlflow_params` (optional): Extra keyword arguments forwarded to `sktime.utils.mlflow_sktime.save_model`
245+
246+
**Example:**
247+
```json
248+
{
249+
"estimator_handle": "est_abc123",
250+
"path": "/absolute/path/to/model_dir",
251+
"mlflow_params": {
252+
"serialization_format": "cloudpickle"
253+
}
254+
}
255+
```
256+
257+
**Returns:** `{"success": true, "saved_path": "/absolute/path/to/model_dir", "message": "Model saved successfully to '/absolute/path/to/model_dir'"}`
258+
259+
**Note:** This tool requires MLflow to be available in the server environment.
260+
261+
---
262+
238263
### Datasets
239264

240-
#### 9. `list_datasets`
265+
#### 10. `list_datasets`
241266
List all available demo datasets for testing and experimentation.
242267

243268
**Arguments:** None
@@ -248,7 +273,7 @@ List all available demo datasets for testing and experimentation.
248273

249274
### Handle Management
250275

251-
#### 10. `list_handles`
276+
#### 11. `list_handles`
252277
List all active estimator handles and their status.
253278

254279
**Arguments:** None
@@ -257,7 +282,7 @@ List all active estimator handles and their status.
257282

258283
---
259284

260-
#### 11. `release_handle`
285+
#### 12. `release_handle`
261286
Release an estimator handle and free memory.
262287

263288
**Arguments:**

docs/usage-examples.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -159,3 +159,23 @@ When you are done with an estimator, it's good practice to release it to free up
159159
}
160160
}
161161
```
162+
163+
### Model Persistence
164+
165+
**Save a Fitted Estimator**
166+
Use `save_model` after fitting an estimator or pipeline handle. The underlying `sktime.utils.mlflow_sktime.save_model` API saves to a local filesystem path.
167+
168+
```json
169+
{
170+
"name": "save_model",
171+
"arguments": {
172+
"estimator_handle": "est_abc123",
173+
"path": "/absolute/path/to/model_dir",
174+
"mlflow_params": {
175+
"serialization_format": "pickle"
176+
}
177+
}
178+
}
179+
```
180+
181+
*Returns:* `{"success": true, "saved_path": "/absolute/path/to/model_dir", "message": "Model saved successfully to '/absolute/path/to/model_dir'"}`

docs/user-guide.md

Lines changed: 46 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@ The `sktime-mcp` server exposes a suite of tools designed for Large Language Mod
5454
| **Instantiation** | `instantiate_estimator`, `instantiate_pipeline` | Create model instances or complex pipelines. |
5555
| **Execution** | `fit_predict`, `fit`, `predict` | Train models and generate forecasts. |
5656
| **Data** | `load_data_source`, `list_datasets` | Load data from Pandas, CSV/Parquet, or SQL. |
57-
| **Export** | `export_code` | Generate Python code to reproduce your results. |
57+
| **Export** | `export_code`, `save_model` | Generate Python code or persist fitted estimators to a local path. |
5858

5959
---
6060

@@ -114,6 +114,46 @@ Check if components work together (e.g., Deseasonalizer -> Detrender -> ARIMA).
114114
}
115115
```
116116

117+
### 3. Save a Fitted Model
118+
119+
Persist a trained estimator to a local filesystem path using sktime's MLflow integration.
120+
121+
**Step 1: Fit the estimator**
122+
```json
123+
{
124+
"tool": "fit_predict",
125+
"arguments": {
126+
"estimator_handle": "est_abc123",
127+
"dataset": "airline",
128+
"horizon": 12
129+
}
130+
}
131+
```
132+
133+
**Step 2: Save the fitted model**
134+
```json
135+
{
136+
"tool": "save_model",
137+
"arguments": {
138+
"estimator_handle": "est_abc123",
139+
"path": "/absolute/path/to/model_dir",
140+
"mlflow_params": {
141+
"serialization_format": "cloudpickle"
142+
}
143+
}
144+
}
145+
```
146+
147+
**Typical response**
148+
```json
149+
{
150+
"success": true,
151+
"estimator_handle": "est_abc123",
152+
"saved_path": "/absolute/path/to/model_dir",
153+
"message": "Model saved successfully to '/absolute/path/to/model_dir'"
154+
}
155+
```
156+
117157
---
118158

119159
## 💾 Data Management
@@ -150,6 +190,7 @@ Bring your own data into the MCP server.
150190

151191
- **Resource Management**: Explicitly release handles (`release_handle`, `release_data_handle`) when done to free up memory.
152192
- **Reproducibility**: Always use `export_code` after a successful experiment to save your work.
193+
- **Persistence**: Use `save_model` after fitting if you need the estimator to survive server restarts.
153194
- **Data Hygiene**: Use `auto_format_on_load` for messy real-world data to avoid frequent validation errors.
154195

155196
---
@@ -158,9 +199,9 @@ Bring your own data into the MCP server.
158199

159200
While `sktime-mcp` is a powerful tool for prototyping, please be aware of the current architectural limitations.
160201

161-
#### 1. In-Memory "Amnesia" (No Persistence)
162-
The server stores state in standard Python dictionaries.
163-
> **Impact**: If the server restarts or connection drops, all loaded data and trained models are lost. There is no disk-backed checkpointing.
202+
#### 1. In-Memory Handles (Explicit Persistence Required)
203+
The server stores active handles in standard Python dictionaries.
204+
> **Impact**: If the server restarts or connection drops, in-memory handles are lost. Use `save_model` to persist fitted estimators to a local filesystem path when needed.
164205
165206
#### 2. Synchronous Execution (GIL Blocking)
166207
Heavy operations (like `AutoARIMA` fitting) run on the main thread.
@@ -197,5 +238,6 @@ Complex sktime types (Periods, Intervals) are converted to strings for LLM consu
197238
|-------|----------|
198239
| **"Unknown estimator"** | Use `search_estimators` to find the exact case-sensitive name. |
199240
| **"Missing dependencies"** | Run `pip install -e ".[all]"` to ensure all extras are present. |
241+
| **`save_model` import/runtime errors** | Install MLflow in the environment used by the server. The tool relies on `sktime.utils.mlflow_sktime.save_model` and saves to a local filesystem path. |
200242
| **Validation Failures** | Enable `auto_format_on_load` or use `format_time_series` to clean your data. |
201243
| **Server Timeout** | Heavy models take time. Be patient or try a simpler model (e.g., `NaiveForecaster`) first. |

src/sktime_mcp/server.py

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,7 @@
5656
delete_job_tool,
5757
cleanup_old_jobs_tool,
5858
)
59+
from sktime_mcp.tools.save_model import save_model_tool
5960
from sktime_mcp.composition.validator import get_composition_validator
6061

6162
# Configure logging to stderr with detailed format
@@ -486,6 +487,28 @@ async def list_tools() -> List[Tool]:
486487
},
487488
},
488489
),
490+
Tool(
491+
name="save_model",
492+
description="Save an estimator/pipeline handle using sktime MLflow integration",
493+
inputSchema={
494+
"type": "object",
495+
"properties": {
496+
"estimator_handle": {
497+
"type": "string",
498+
"description": "Handle ID of the estimator to save",
499+
},
500+
"path": {
501+
"type": "string",
502+
"description": "Local directory or URI where the model will be saved",
503+
},
504+
"mlflow_params": {
505+
"type": "object",
506+
"description": "Optional extra parameters for sktime.utils.mlflow_sktime.save_model",
507+
},
508+
},
509+
"required": ["estimator_handle", "path"],
510+
},
511+
),
489512
]
490513

491514

@@ -589,6 +612,12 @@ async def call_tool(name: str, arguments: Dict[str, Any]) -> List[TextContent]:
589612
result = delete_job_tool(arguments["job_id"])
590613
elif name == "cleanup_old_jobs":
591614
result = cleanup_old_jobs_tool(arguments.get("max_age_hours", 24))
615+
elif name == "save_model":
616+
result = save_model_tool(
617+
arguments["estimator_handle"],
618+
arguments["path"],
619+
arguments.get("mlflow_params"),
620+
)
592621
else:
593622
result = {"error": f"Unknown tool: {name}"}
594623

src/sktime_mcp/tools/__init__.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
from sktime_mcp.tools.instantiate import instantiate_estimator_tool
66
from sktime_mcp.tools.fit_predict import fit_predict_tool
77
from sktime_mcp.tools.codegen import export_code_tool
8+
from sktime_mcp.tools.save_model import save_model_tool
89
from sktime_mcp.tools.format_tools import (
910
format_time_series_tool,
1011
auto_format_on_load_tool,
@@ -16,6 +17,7 @@
1617
"instantiate_estimator_tool",
1718
"fit_predict_tool",
1819
"export_code_tool",
20+
"save_model_tool",
1921
"format_time_series_tool",
2022
"auto_format_on_load_tool",
2123
]

src/sktime_mcp/tools/save_model.py

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
"""
2+
save_model tool for sktime MCP.
3+
4+
Saves estimator instances via sktime's MLflow integration.
5+
"""
6+
7+
from typing import Any, Callable, Dict, Optional
8+
9+
from sktime_mcp.runtime.handles import get_handle_manager
10+
11+
12+
def _get_mlflow_save_model() -> Callable[..., Any]:
13+
"""Resolve sktime MLflow save utility lazily for better runtime compatibility."""
14+
try:
15+
from sktime.utils.mlflow_sktime import save_model as mlflow_save_model
16+
except Exception as exc:
17+
raise ImportError(
18+
"Unable to import sktime MLflow save_model utility. "
19+
"Ensure sktime and MLflow dependencies are installed."
20+
) from exc
21+
return mlflow_save_model
22+
23+
24+
def save_model_tool(
25+
estimator_handle: str,
26+
path: str,
27+
mlflow_params: Optional[Dict[str, Any]] = None,
28+
) -> Dict[str, Any]:
29+
"""
30+
Save an instantiated estimator to a local path or URI using sktime+MLflow.
31+
32+
Args:
33+
estimator_handle: Handle ID from instantiate_estimator / instantiate_pipeline
34+
path: Local directory or URI where the model should be saved
35+
mlflow_params: Optional extra keyword arguments for sktime MLflow save_model
36+
37+
Returns:
38+
Dictionary with success status and confirmation message/path.
39+
"""
40+
handle_manager = get_handle_manager()
41+
42+
try:
43+
estimator = handle_manager.get_instance(estimator_handle)
44+
except KeyError:
45+
return {"success": False, "error": f"Handle not found: {estimator_handle}"}
46+
47+
if mlflow_params is not None and not isinstance(mlflow_params, dict):
48+
return {"success": False, "error": "mlflow_params must be a dictionary"}
49+
50+
try:
51+
save_model = _get_mlflow_save_model()
52+
save_model(sktime_model=estimator, path=path, **(mlflow_params or {}))
53+
return {
54+
"success": True,
55+
"estimator_handle": estimator_handle,
56+
"saved_path": path,
57+
"message": f"Model saved successfully to '{path}'",
58+
}
59+
except Exception as exc:
60+
return {"success": False, "error": str(exc)}

tests/test_core.py

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -153,6 +153,36 @@ def test_describe_unknown_estimator(self):
153153
assert not result["success"]
154154
assert "error" in result
155155

156+
def test_save_model_tool(self, monkeypatch, tmp_path):
157+
"""Test save_model tool resolves handle and forwards parameters."""
158+
from sktime_mcp.runtime.handles import get_handle_manager
159+
from sktime_mcp.tools.save_model import save_model_tool
160+
import sktime_mcp.tools.save_model as save_model_module
161+
162+
calls = {}
163+
164+
def fake_save_model(**kwargs):
165+
calls.update(kwargs)
166+
167+
monkeypatch.setattr(save_model_module, "_get_mlflow_save_model", lambda: fake_save_model)
168+
169+
handle_manager = get_handle_manager()
170+
handle = handle_manager.create_handle("DummyEstimator", object())
171+
172+
try:
173+
result = save_model_tool(
174+
estimator_handle=handle,
175+
path=str(tmp_path / "model_dir"),
176+
mlflow_params={"serialization_format": "pickle"},
177+
)
178+
finally:
179+
handle_manager.release_handle(handle)
180+
181+
assert result["success"]
182+
assert result["saved_path"] == str(tmp_path / "model_dir")
183+
assert calls["path"] == str(tmp_path / "model_dir")
184+
assert calls["serialization_format"] == "pickle"
185+
156186

157187
if __name__ == "__main__":
158188
pytest.main([__file__, "-v"])

0 commit comments

Comments
 (0)