Skip to content

Commit a9f57e7

Browse files
VShturmaodorfer
andauthored
feat: add MLflow Prompt Registry export/import support (#5)
* feat: add MLflow Prompt Registry export/import support * fix: add newline * fix: add small fixes * fix: types * fix: replace union * fix: typo * fix: types * fix: types * fix: add import * fix: typo --------- Co-authored-by: odorfer <odorfer@amazon.de>
1 parent 5ed5d16 commit a9f57e7

File tree

15 files changed

+1491
-3
lines changed

15 files changed

+1491
-3
lines changed

docs/bulk.md

Lines changed: 86 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,8 @@ Notes:
2424
| | [import-models](#Import-registered-models) | [code](mlflow_migration/bulk/import_models.py) | Imports registered models from a directory. |
2525
| Experiment | [export-experiments](#Export-experiments) | [code](mlflow_migration/bulk/export_experiments.py) | Export several (or all) experiments to a directory. |
2626
| | [import-experiments](#Import-experiments) | [code](mlflow_migration/bulk/import_experiments.py) | Imports experiments from a directory. |
27-
27+
| Prompt | [export-prompts](#Export-prompts) | [code](mlflow_migration/bulk/export_prompts.py) | Export prompts from the MLflow Prompt Registry (MLflow 2.21.0+). |
28+
| | [import-prompts](#Import-prompts) | [code](mlflow_migration/bulk/import_prompts.py) | Imports prompts to the MLflow Prompt Registry. |
2829

2930
## All MLflow Objects Tools
3031

@@ -387,3 +388,87 @@ cat experiment-names.csv
387388
/Users/me@mycompany.com,/Users/you@mycompany.com
388389
/Users/foo@mycompany.com,/Users/bar@mycompany.com
389390
```
391+
392+
## Prompts
393+
394+
Export/import prompts from the MLflow Prompt Registry (MLflow 2.21.0+).
395+
396+
**Notes:**
397+
* Prompt Registry support requires MLflow 2.21.0 or higher. The export/import will be skipped with a warning message if the MLflow version doesn't support prompts.
398+
* **Version handling**: MLflow 3.0+ uses efficient pagination to export all prompt versions. MLflow 2.21-2.x uses iterative discovery with early stopping (stops after 3 consecutive missing versions).
399+
400+
### Export prompts
401+
402+
Export prompts from the MLflow Prompt Registry to a directory.
403+
404+
Source: [export_prompts.py](mlflow_migration/bulk/export_prompts.py).
405+
406+
#### Usage
407+
408+
```
409+
export-prompts --help
410+
411+
Options:
412+
--output-dir TEXT Output directory. [required]
413+
--prompts TEXT Prompt names: 'all' for all prompts, comma-delimited
414+
list (e.g., 'prompt1,prompt2'), or file path ending
415+
with '.txt' containing prompt names (one per line).
416+
[required]
417+
--use-threads BOOLEAN Use multithreading for export. [default: False]
418+
```
419+
420+
#### Examples
421+
422+
##### Export all prompts
423+
```
424+
export-prompts \
425+
--output-dir out/prompts \
426+
--prompts all
427+
```
428+
429+
##### Export specific prompts
430+
```
431+
export-prompts \
432+
--output-dir out/prompts \
433+
--prompts my-prompt-1,my-prompt-2
434+
```
435+
436+
##### Export prompts from filename
437+
```
438+
export-prompts \
439+
--output-dir out/prompts \
440+
--prompts my-prompts.txt
441+
```
442+
443+
where `my-prompts.txt` is:
444+
```
445+
my-prompt-1
446+
my-prompt-2
447+
```
448+
449+
### Import prompts
450+
451+
Import prompts to the MLflow Prompt Registry from a directory.
452+
453+
Source: [import_prompts.py](mlflow_migration/bulk/import_prompts.py).
454+
455+
#### Usage
456+
457+
```
458+
import-prompts --help
459+
460+
Options:
461+
--input-dir TEXT Input directory containing exported prompts. [required]
462+
--delete-prompt BOOLEAN Delete existing prompt before importing. [default: False]
463+
--use-threads BOOLEAN Use multithreading for import. [default: False]
464+
```
465+
466+
#### Examples
467+
468+
```
469+
import-prompts --input-dir out/prompts
470+
```
471+
472+
**Notes:**
473+
* Prompts are imported with their original names and version numbers are preserved.
474+
* All versions of each prompt are exported and imported to maintain complete version history.

docs/single.md

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,8 @@ See sample JSON export files [here](export_format.md#sample-export-json-files).
2323
| Run | [export-run](#export-run) | [code](mlflow_migration/run/export_run.py) |
2424
| | [import-run](#import-run) | [code](mlflow_migration/run/import_run.py) |
2525
| | [copy-run](copy.md#copy-run) | [code](mlflow_migration/copy/copy_run.py) |
26+
| Prompt | [export-prompt](#export-prompt) | [code](mlflow_migration/prompt/export_prompt.py) |
27+
| | [import-prompt](#import-prompt) | [code](mlflow_migration/prompt/import_prompt.py) |
2628

2729
## Experiment Tools
2830

@@ -493,3 +495,66 @@ Options:
493495
metadata (description and tags). [default:
494496
False]
495497
```
498+
499+
## Prompt Tools
500+
501+
Export and import prompts from the MLflow Prompt Registry (MLflow 2.21.0+).
502+
503+
**Note:** Prompt Registry support requires MLflow 2.21.0 or higher.
504+
505+
### Export Prompt
506+
507+
Export a single prompt version to a directory.
508+
509+
Source: [export_prompt.py](mlflow_migration/prompt/export_prompt.py).
510+
511+
#### Usage
512+
513+
```
514+
export-prompt --help
515+
516+
Options:
517+
--prompt-name TEXT Name of the prompt to export. [required]
518+
--prompt-version TEXT Version of the prompt to export. [required]
519+
--output-dir TEXT Output directory. [required]
520+
```
521+
522+
#### Example
523+
524+
```
525+
export-prompt \
526+
--prompt-name my-greeting-prompt \
527+
--prompt-version 1 \
528+
--output-dir out
529+
```
530+
531+
### Import Prompt
532+
533+
Import a prompt from an exported directory.
534+
535+
Source: [import_prompt.py](mlflow_migration/prompt/import_prompt.py).
536+
537+
#### Usage
538+
539+
```
540+
import-prompt --help
541+
542+
Options:
543+
--input-dir TEXT Input directory containing exported prompt. [required]
544+
--prompt-name TEXT Optional new name for the imported prompt. If not
545+
specified, uses original name.
546+
```
547+
548+
#### Examples
549+
550+
##### Import with original name
551+
```
552+
import-prompt --input-dir out
553+
```
554+
555+
##### Import with new name
556+
```
557+
import-prompt \
558+
--input-dir out \
559+
--prompt-name my-new-prompt-name
560+
```

mlflow_migration/bulk/export_all.py

Lines changed: 27 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@
2323
from mlflow_migration.client.client_utils import create_mlflow_client
2424
from mlflow_migration.bulk.export_models import export_models
2525
from mlflow_migration.bulk.export_experiments import export_experiments
26+
from mlflow_migration.bulk.export_prompts import export_prompts
2627

2728
ALL_STAGES = "Production,Staging,Archived,None"
2829

@@ -73,6 +74,26 @@ def export_all(
7374
notebook_formats=notebook_formats,
7475
use_threads=use_threads,
7576
)
77+
78+
# Export prompts (returns dict with status)
79+
res_prompts = None
80+
try:
81+
_logger.info("Exporting prompts...")
82+
res_prompts = export_prompts(
83+
output_dir=os.path.join(output_dir, "prompts"),
84+
prompt_names=None, # Export all prompts
85+
use_threads=use_threads,
86+
mlflow_client=mlflow_client
87+
)
88+
# Log if unsupported but don't fail
89+
if res_prompts and "unsupported" in res_prompts:
90+
_logger.warning(f"Prompts not supported in MLflow {res_prompts.get('mlflow_version')}")
91+
elif res_prompts and "error" in res_prompts:
92+
_logger.warning(f"Failed to export prompts: {res_prompts['error']}")
93+
except Exception as e:
94+
_logger.warning(f"Failed to export prompts: {e}")
95+
res_prompts = {"error": str(e)}
96+
7697
duration = round(time.time() - start_time, 1)
7798
info_attr = {
7899
"options": {
@@ -83,7 +104,12 @@ def export_all(
83104
"use_threads": use_threads,
84105
"output_dir": output_dir,
85106
},
86-
"status": {"duration": duration, "models": res_models, "experiments": res_exps},
107+
"status": {
108+
"duration": duration,
109+
"models": res_models,
110+
"experiments": res_exps,
111+
"prompts": res_prompts
112+
}
87113
}
88114
io_utils.write_export_file(output_dir, "manifest.json", __file__, {}, info_attr)
89115
_logger.info(f"Duration for entire tracking server export: {duration} seconds")

0 commit comments

Comments
 (0)