Skip to content

Commit d3b8de7

Browse files
author
Larry Franks
committed
documenting v1 known issue
1 parent d9e61dd commit d3b8de7

File tree

1 file changed

+80
-0
lines changed

1 file changed

+80
-0
lines changed

articles/machine-learning/v1/how-to-debug-pipelines.md

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -267,6 +267,86 @@ For more information on using the OpenCensus Python library in this manner, see
267267

268268
In some cases, you may need to interactively debug the Python code used in your ML pipeline. By using Visual Studio Code (VS Code) and debugpy, you can attach to the code as it runs in the training environment. For more information, visit the [interactive debugging in VS Code guide](../how-to-debug-visual-studio-code.md#debug-and-troubleshoot-machine-learning-pipelines).
269269

270+
## HyperdriveStep and AutoMLStep fail with network isolation
271+
272+
After using HyperdriveStep and AutoMLStep, when you attempt to register the model you may receive an error.
273+
274+
* You are using Azure Machine Learning SDK v1.
275+
* Your Azure Machine Learning workspace is configured for network isolation (VNet).
276+
* Your pipeline attempts to register the model generated by the previous step. For example, in the following code the `inputs` parameter is the saved_model from a HyperdriveStep:
277+
278+
```python
279+
register_model_step = PythonScriptStep(script_name='register_model.py',
280+
name="register_model_step01",
281+
inputs=[saved_model],
282+
compute_target=cpu_cluster,
283+
arguments=["--saved-model", saved_model],
284+
allow_reuse=True,
285+
runconfig=rcfg)
286+
```
287+
288+
### Workaround
289+
290+
> [!IMPORTANT]
291+
> This behavior does not occur when using Azure Machine Learning SDK v2.
292+
293+
To work around this error, use the [Run](/python/api/azureml-core/azureml.core.run.run) class to get the model created from the HyperdriveStep or AutoMLStep. The following is an example script that gets the output model from a HyperdriveStep:
294+
295+
```python
296+
%%writefile $script_folder/model_download9.py
297+
import argparse
298+
from azureml.core import Run
299+
from azureml.pipeline.core import PipelineRun
300+
from azureml.core.experiment import Experiment
301+
from azureml.train.hyperdrive import HyperDriveRun
302+
from azureml.pipeline.steps import HyperDriveStepRun
303+
304+
if __name__ == "__main__":
305+
parser = argparse.ArgumentParser()
306+
parser.add_argument(
307+
'--hd_step_name',
308+
type=str, dest='hd_step_name',
309+
help='The name of the step that runs AutoML training within this pipeline')
310+
311+
312+
313+
args = parser.parse_args()
314+
315+
current_run = Run.get_context()
316+
317+
pipeline_run = PipelineRun(current_run.experiment, current_run.experiment.name)
318+
319+
hd_step_run = HyperDriveStepRun((pipeline_run.find_step_run(args.hd_step_name))[0])
320+
hd_best_run = hd_step_run.get_best_run_by_primary_metric()
321+
322+
print(hd_best_run)
323+
hd_best_run.download_file("outputs/model/saved_model.pb", "saved_model.pb")
324+
325+
326+
print("Successfully downloaded model")
327+
```
328+
329+
The file can then be used from a PythonScriptStep:
330+
331+
```python
332+
from azureml.pipeline.steps import PythonScriptStep
333+
conda_dep = CondaDependencies()
334+
conda_dep.add_pip_package("azureml-sdk")
335+
conda_dep.add_pip_package("azureml-pipeline")
336+
337+
rcfg = RunConfiguration(conda_dependencies=conda_dep)
338+
339+
model_download_step = PythonScriptStep(
340+
name="Download Model 9",
341+
script_name="model_download9.py",
342+
arguments=["--hd_step_name", hd_step_name],
343+
compute_target=compute_target,
344+
source_directory=script_folder,
345+
allow_reuse=False,
346+
runconfig=rcfg
347+
)
348+
```
349+
270350
## Next steps
271351

272352
* For a complete tutorial using `ParallelRunStep`, see [Tutorial: Build an Azure Machine Learning pipeline for batch scoring](../tutorial-pipeline-batch-scoring-classification.md).

0 commit comments

Comments
 (0)