Skip to content

Commit 2b2a822

Browse files
authored
Merge pull request #204584 from santiagxf/santiagxf/aml-mlflow-runs-fix
AML mlflow runs fix
2 parents 40bdb02 + 6df5ff8 commit 2b2a822

File tree

5 files changed

+122
-104
lines changed

5 files changed

+122
-104
lines changed

articles/machine-learning/concept-mlflow-models.md

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -157,10 +157,19 @@ name: mlflow-env
157157

158158
### Model's predict function
159159

160-
All MLflow models contain a `predict` function. This function is the one that is called when a model is deployed using a no-code-deployment experience. What the `predict` function returns (classes, probabilities, a forecast, etc.) depend on the framework (i.e. flavor) used for training. Read the documentation of each flavor to know what they return.
160+
All MLflow models contain a `predict` function. **This function is the one that is called when a model is deployed using a no-code-deployment experience**. What the `predict` function returns (classes, probabilities, a forecast, etc.) depend on the framework (i.e. flavor) used for training. Read the documentation of each flavor to know what they return.
161161

162162
In same cases, you may need to customize this function to change the way inference is executed. On those cases, you will need to [log models with a different behavior in the predict method](how-to-log-mlflow-models.md#logging-models-with-a-different-behavior-in-the-predict-method) or [log a custom model's flavor](how-to-log-mlflow-models.md#logging-custom-models).
163163

164+
## Loading MLflow models back
165+
166+
Models created as MLflow models can be loaded back directly from the run where they were logged, from the file system where they are saved or from the model registry where they are registered. MLflow provides a consistent way to load those models regardless of the location.
167+
168+
There are two workflows available for loading models:
169+
170+
* **Loading back the same object and types that were logged:**: You can load models using MLflow SDK and obtain an instance of the model with types belonging to the training library. For instance, an ONNX model will return a `ModelProto` while a decision tree trained with Scikit-Learn model will return a `DecisionTreeClassifier` object. Use `mlflow.<flavor>.load_model()` to do so.
171+
* **Loading back a model for running inference:** You can load models using MLflow SDK and obtain a wrapper where MLflow warranties there will be a `predict` function. It doesn't matter which flavor you are using, every MLflow model needs to implement this contract. Furthermore, MLflow warranties that this function can be called using arguments of type `pandas.DataFrame`, `numpy.ndarray` or `dict[strin, numpyndarray]` (depending on the signature of the model). MLflow handles the type conversion to the input type the model actually expects. Use `mlflow.pyfunc.load_model()` to do so.
172+
164173
## Start logging models
165174

166175
We recommend starting taking advantage of MLflow models in Azure Machine Learning. There are different ways to start using the model's concept with MLflow. Read [How to log MLFlow models](how-to-log-mlflow-models.md) to a comprehensive guide.

articles/machine-learning/how-to-log-mlflow-models.md

Lines changed: 82 additions & 87 deletions
Original file line numberDiff line numberDiff line change
@@ -42,14 +42,13 @@ import mlflow
4242
from xgboost import XGBClassifier
4343
from sklearn.metrics import accuracy_score
4444

45-
with mlflow.start_run():
46-
mlflow.autolog()
45+
mlflow.autolog()
4746

48-
model = XGBClassifier(use_label_encoder=False, eval_metric="logloss")
49-
model.fit(X_train, y_train, eval_set=[(X_test, y_test)], verbose=False)
50-
y_pred = model.predict(X_test)
47+
model = XGBClassifier(use_label_encoder=False, eval_metric="logloss")
48+
model.fit(X_train, y_train, eval_set=[(X_test, y_test)], verbose=False)
5149

52-
accuracy = accuracy_score(y_test, y_pred)
50+
y_pred = model.predict(X_test)
51+
accuracy = accuracy_score(y_test, y_pred)
5352
```
5453

5554
> [!TIP]
@@ -76,34 +75,33 @@ from sklearn.metrics import accuracy_score
7675
from mlflow.models import infer_signature
7776
from mlflow.utils.environment import _mlflow_conda_env
7877

79-
with mlflow.start_run():
80-
mlflow.autolog(log_models=False)
81-
82-
model = XGBClassifier(use_label_encoder=False, eval_metric="logloss")
83-
model.fit(X_train, y_train, eval_set=[(X_test, y_test)], verbose=False)
84-
y_pred = model.predict(X_test)
85-
86-
accuracy = accuracy_score(y_test, y_pred)
87-
88-
# Signature
89-
signature = infer_signature(X_test, y_test)
90-
91-
# Conda environment
92-
custom_env =_mlflow_conda_env(
93-
additional_conda_deps=None,
94-
additional_pip_deps=["xgboost==1.5.2"],
95-
additional_conda_channels=None,
96-
)
97-
98-
# Sample
99-
input_example = X_train.sample(n=1)
100-
101-
# Log the model manually
102-
mlflow.xgboost.log_model(model,
103-
artifact_path="classifier",
104-
conda_env=custom_env,
105-
signature=signature,
106-
input_example=input_example)
78+
mlflow.autolog(log_models=False)
79+
80+
model = XGBClassifier(use_label_encoder=False, eval_metric="logloss")
81+
model.fit(X_train, y_train, eval_set=[(X_test, y_test)], verbose=False)
82+
y_pred = model.predict(X_test)
83+
84+
accuracy = accuracy_score(y_test, y_pred)
85+
86+
# Signature
87+
signature = infer_signature(X_test, y_test)
88+
89+
# Conda environment
90+
custom_env =_mlflow_conda_env(
91+
additional_conda_deps=None,
92+
additional_pip_deps=["xgboost==1.5.2"],
93+
additional_conda_channels=None,
94+
)
95+
96+
# Sample
97+
input_example = X_train.sample(n=1)
98+
99+
# Log the model manually
100+
mlflow.xgboost.log_model(model,
101+
artifact_path="classifier",
102+
conda_env=custom_env,
103+
signature=signature,
104+
input_example=input_example)
107105
```
108106

109107
> [!NOTE]
@@ -166,20 +164,19 @@ from xgboost import XGBClassifier
166164
from sklearn.metrics import accuracy_score
167165
from mlflow.models import infer_signature
168166

169-
with mlflow.start_run():
170-
mlflow.xgboost.autolog(log_models=False)
167+
mlflow.xgboost.autolog(log_models=False)
171168

172-
model = XGBClassifier(use_label_encoder=False, eval_metric="logloss")
173-
model.fit(X_train, y_train, eval_set=[(X_test, y_test)], verbose=False)
174-
y_probs = model.predict_proba(X_test)
169+
model = XGBClassifier(use_label_encoder=False, eval_metric="logloss")
170+
model.fit(X_train, y_train, eval_set=[(X_test, y_test)], verbose=False)
171+
y_probs = model.predict_proba(X_test)
175172

176-
accuracy = accuracy_score(y_test, y_probs.argmax(axis=1))
177-
mlflow.log_metric("accuracy", accuracy)
173+
accuracy = accuracy_score(y_test, y_probs.argmax(axis=1))
174+
mlflow.log_metric("accuracy", accuracy)
178175

179-
signature = infer_signature(X_test, y_probs)
180-
mlflow.pyfunc.log_model("classifier",
181-
python_model=ModelWrapper(model),
182-
signature=signature)
176+
signature = infer_signature(X_test, y_probs)
177+
mlflow.pyfunc.log_model("classifier",
178+
python_model=ModelWrapper(model),
179+
signature=signature)
183180
```
184181

185182
> [!TIP]
@@ -248,33 +245,32 @@ from sklearn.preprocessing import OrdinalEncoder
248245
from sklearn.metrics import accuracy_score
249246
from mlflow.models import infer_signature
250247

251-
with mlflow.start_run():
252-
mlflow.xgboost.autolog(log_models=False)
253-
254-
encoder = OrdinalEncoder(handle_unknown='ignore')
255-
X_train['thal'] = enc.fit_transform(X_train['thal'])
256-
X_test['thal'] = enc.transform(X_test['thal'])
257-
258-
model = XGBClassifier(use_label_encoder=False, eval_metric="logloss")
259-
model.fit(X_train, y_train, eval_set=[(X_test, y_test)], verbose=False)
260-
y_probs = model.predict_proba(X_test)
261-
262-
accuracy = accuracy_score(y_test, y_probs.argmax(axis=1))
263-
mlflow.log_metric("accuracy", accuracy)
264-
265-
encoder_path = 'encoder.pkl'
266-
joblib.dump(encoder, encoder_path)
267-
model_path = "xgb.model"
268-
model.save_model(model_path)
269-
270-
signature = infer_signature(X, y_probs)
271-
mlflow.pyfunc.log_model("classifier",
272-
python_model=ModelWrapper(),
273-
artifacts={
274-
'encoder': encoder_path,
275-
'model': model_path
276-
},
277-
signature=signature)
248+
mlflow.xgboost.autolog(log_models=False)
249+
250+
encoder = OrdinalEncoder(handle_unknown='ignore')
251+
X_train['thal'] = enc.fit_transform(X_train['thal'])
252+
X_test['thal'] = enc.transform(X_test['thal'])
253+
254+
model = XGBClassifier(use_label_encoder=False, eval_metric="logloss")
255+
model.fit(X_train, y_train, eval_set=[(X_test, y_test)], verbose=False)
256+
y_probs = model.predict_proba(X_test)
257+
258+
accuracy = accuracy_score(y_test, y_probs.argmax(axis=1))
259+
mlflow.log_metric("accuracy", accuracy)
260+
261+
encoder_path = 'encoder.pkl'
262+
joblib.dump(encoder, encoder_path)
263+
model_path = "xgb.model"
264+
model.save_model(model_path)
265+
266+
signature = infer_signature(X, y_probs)
267+
mlflow.pyfunc.log_model("classifier",
268+
python_model=ModelWrapper(),
269+
artifacts={
270+
'encoder': encoder_path,
271+
'model': model_path
272+
},
273+
signature=signature)
278274
```
279275

280276
# [Using a model loader](#tab/loader)
@@ -340,25 +336,24 @@ from xgboost import XGBClassifier
340336
from sklearn.metrics import accuracy_score
341337
from mlflow.models import infer_signature
342338

343-
with mlflow.start_run():
344-
mlflow.xgboost.autolog(log_models=False)
339+
mlflow.xgboost.autolog(log_models=False)
345340

346-
model = XGBClassifier(use_label_encoder=False, eval_metric="logloss")
347-
model.fit(X_train, y_train, eval_set=[(X_test, y_test)], verbose=False)
348-
y_probs = model.predict_proba(X_test)
341+
model = XGBClassifier(use_label_encoder=False, eval_metric="logloss")
342+
model.fit(X_train, y_train, eval_set=[(X_test, y_test)], verbose=False)
343+
y_probs = model.predict_proba(X_test)
349344

350-
accuracy = accuracy_score(y_test, y_probs.argmax(axis=1))
351-
mlflow.log_metric("accuracy", accuracy)
345+
accuracy = accuracy_score(y_test, y_probs.argmax(axis=1))
346+
mlflow.log_metric("accuracy", accuracy)
352347

353-
model_path = "xgb.model"
354-
model.save_model(model_path)
348+
model_path = "xgb.model"
349+
model.save_model(model_path)
355350

356-
signature = infer_signature(X_test, y_probs)
357-
mlflow.pyfunc.log_model("classifier",
358-
data_path=model_path,
359-
code_path=["loader_module.py"],
360-
loader_module="loader_module",
361-
signature=signature)
351+
signature = infer_signature(X_test, y_probs)
352+
mlflow.pyfunc.log_model("classifier",
353+
data_path=model_path,
354+
code_path=["loader_module.py"],
355+
loader_module="loader_module",
356+
signature=signature)
362357
```
363358

364359
---

articles/machine-learning/how-to-manage-models-mlflow.md

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -90,7 +90,7 @@ mlflow.register_model(f"file://{model_local_path}", "local-model-test")
9090
> [!NOTE]
9191
> Notice how the model URI schema `file:/` requires absolute paths.
9292
93-
## Querying models
93+
## Querying model registries
9494

9595
### Querying all the models in the registry
9696

@@ -123,6 +123,17 @@ If you need a specific version of the model, you can indicate so:
123123
client.get_model_version(model_name, version=2)
124124
```
125125

126+
## Loading models from registry
127+
128+
You can load models directly from the registry to restore the models objects that were logged. Use the functions `mlflow.<flavor>.load_model()` or `mlflow.pyfunc.load_model()` indicating the URI of the model you want to load using the following syntax:
129+
130+
* `models:/<model-name>/latest`, to load the last version of the model.
131+
* `models:/<model-name>/<version-number>`, to load a specific version of the model.
132+
* `models:/<model-name>/<stage-name>`, to load a specific version in a given stage for a model. View [Model stages](#model-stages) for details.
133+
134+
> [!TIP]
135+
> For learning about the difference between `mlflow.<flavor>.load_model()` and `mlflow.pyfunc.load_model()`, view [Loading MLflow models back](concept-mlflow-models.md#loading-mlflow-models-back) article.
136+
126137
## Model stages
127138

128139
MLflow supports model's stages to manage model's lifecycle. Model's version can transition from one stage to another. Stages are assigned to a model's version (instead of models) which means that a given model can have multiple versions on different stages.

articles/machine-learning/how-to-track-experiments-mlflow.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -251,6 +251,9 @@ MLflow also allows you to both operations at once and download and load the mode
251251
model = mlflow.xgboost.load_model(f"runs:/{last_run.info.run_id}/{artifact_path}")
252252
```
253253

254+
> [!TIP]
255+
> You can also load models from the registry using MLflow. View [loading MLflow models with MLflow](how-to-manage-models-mlflow.md#loading-models-from-registry) for details.
256+
254257
## Getting child (nested) runs
255258

256259
MLflow supports the concept of child (nested) runs. They are useful when you need to spin off training routines requiring being tracked independently from the main training process. This is the typical case of hyper-parameter tuning for instance. You can query all the child runs of a specific run using the property tag `mlflow.parentRunId`, which contains the run ID of the parent run.

articles/machine-learning/toc.yml

Lines changed: 15 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -414,28 +414,28 @@
414414
href: how-to-migrate-from-estimators-to-scriptrunconfig.md
415415
- name: Reinforcement learning
416416
href: how-to-use-reinforcement-learning.md
417-
- name: Track & monitor training
418-
items:
419-
- name: Monitor training jobs
420-
displayName: cancel, fail, status, child run
421-
href: how-to-track-monitor-analyze-runs.md
422-
- name: Log & view metrics, parameters and files
423-
displayName: troubleshoot, log, files, tracing, metrics
424-
href: how-to-log-view-metrics.md
425-
- name: Log MLflow models
426-
href: how-to-log-mlflow-models.md
427-
- name: Visualize runs with TensorBoard
428-
displayName: log, monitor, metrics
429-
href: how-to-monitor-tensorboard.md
430-
- name: Migrate from SDK v1 logging to MLflow
431-
href: reference-migrate-sdk-v1-mlflow-tracking.md
432417
- name: Use Key Vault when training
433418
displayName: secrets keyvault
434419
href: how-to-use-secrets-in-runs.md
435420
- name: Train with the CLI v2
436421
href: how-to-train-cli.md
437422
- name: Train with the REST API
438423
href: how-to-train-with-rest.md
424+
- name: Track & monitor training
425+
items:
426+
- name: Monitor training jobs
427+
displayName: cancel, fail, status, child run
428+
href: how-to-track-monitor-analyze-runs.md
429+
- name: Log & view metrics, parameters and files
430+
displayName: troubleshoot, log, files, tracing, metrics
431+
href: how-to-log-view-metrics.md
432+
- name: Log MLflow models
433+
href: how-to-log-mlflow-models.md
434+
- name: Visualize runs with TensorBoard
435+
displayName: log, monitor, metrics
436+
href: how-to-monitor-tensorboard.md
437+
- name: Migrate from SDK v1 logging to MLflow
438+
href: reference-migrate-sdk-v1-mlflow-tracking.md
439439
- name: Train and track with MLflow
440440
items:
441441
- name: Track experiments with MLflow

0 commit comments

Comments
 (0)