Merge pull request #110623 from sdgilley/sdg-local-remote

denrea · web-flow · commit e1ac60f5ca55 · 2020-04-30T12:01:27.000-07:00
Automated ML local vs. remote
diff --git a/articles/machine-learning/concept-automated-ml.md b/articles/machine-learning/concept-automated-ml.md
@@ -130,11 +130,101 @@ Enable this setting with:
 
 + Python SDK: Specifying `"feauturization": 'auto' / 'off' / 'FeaturizationConfig'` for the [`AutoMLConfig` class](/python/api/azureml-train-automl-client/azureml.train.automl.automlconfig.automlconfig). 
 
-<a name="parity"></a>
+## Classification & regression
+
+Classification and regression are the most common types of machine learning tasks. Both are types of supervised learning in which models learn using training data, and apply those learnings to new data. Azure Machine Learning offers featurizations specifically for these tasks, such as deep neural network text featurizers for classification. Learn more about [featurization options](how-to-use-automated-ml-for-ml-models.md#featurization). 
+
+The main goal of classification models is to predict which categories new data will fall into based on learnings from its training data. Common classification examples include fraud detection, handwriting recognition, and object detection.  Learn more and see an example of [classification with automated machine learning](tutorial-train-models-with-aml.md).
+
+Different from classification where predicted output values are categorical, regression models predict numerical output values based on independent predictors. In regression, the objective is to help establish the relationship among those independent predictor variables by estimating how one variable impacts the others. For example, automobile price based on features like, gas mileage, safety rating, etc. Learn more and see an example of [regression with automated machine learning](tutorial-auto-train-models.md).
+
+## Time-series forecasting
+
+Building forecasts is an integral part of any business, whether it's revenue, inventory, sales, or customer demand. You can use automated ML to combine techniques and approaches and get a recommended, high-quality time-series forecast.
+
+An automated time-series experiment is treated as a multivariate regression problem. Past time-series values are "pivoted" to become additional dimensions for the regressor together with other predictors. This approach, unlike classical time series methods, has an advantage of naturally incorporating multiple contextual variables and their relationship to one another during training. Automated ML learns a single, but often internally branched model for all items in the dataset and prediction horizons. More data is thus available to estimate model parameters and generalization to unseen series becomes possible.
+
+Learn more and see an example of [automated machine learning for time series forecasting](how-to-auto-train-forecast.md). Or, see the [energy demand notebook](https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/automated-machine-learning/forecasting-energy-demand/auto-ml-forecasting-energy-demand.ipynb) for detailed code examples of advanced forecasting configuration including:
+
+* holiday detection and featurization
+* time-series and DNN learners (Auto-ARIMA, Prophet, ForecastTCN)
+* many models support through grouping
+* rolling-origin cross validation
+* configurable lags
+* rolling window aggregate features
+
+## <a name="ensemble"></a> Ensemble models
+
+Automated machine learning supports ensemble models, which are enabled by default. Ensemble learning improves machine learning results and predictive performance by combining multiple models as opposed to using single models. The ensemble iterations appear as the final iterations of your run. Automated machine learning uses both voting and stacking ensemble methods for combining models:
+
+* **Voting**: predicts based on the weighted average of predicted class probabilities (for classification tasks) or predicted regression targets (for regression tasks).
+* **Stacking**: stacking combines heterogenous models and trains a meta-model based on the output from the individual models. The current default meta-models are LogisticRegression for classification tasks and ElasticNet for regression/forecasting tasks.
+
+The [Caruana ensemble selection algorithm](http://www.niculescu-mizil.org/papers/shotgun.icml04.revised.rev2.pdf) with sorted ensemble initialization is used to decide which models to use within the ensemble. At a high level, this algorithm initializes the ensemble with up to five models with the best individual scores, and verifies that these models are within 5% threshold of the best score to avoid a poor initial ensemble. Then for each ensemble iteration, a new model is added to the existing ensemble and the resulting score is calculated. If a new model improved the existing ensemble score, the ensemble is updated to include the new model.
+
+See the [how-to](how-to-configure-auto-train.md#ensemble) for changing default ensemble settings in automated machine learning.
+
+## Use with ONNX 
+
+With Azure Machine Learning, you can use automated ML to build a Python model and have it converted to the ONNX format. Once the models are in the ONNX format, they can be run on a variety of platforms and devices. Learn more about [accelerating ML models with ONNX](concept-onnx.md).
+
+See how to convert to ONNX format [in this Jupyter notebook example](https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/automated-machine-learning/classification-bank-marketing-all-features/auto-ml-classification-bank-marketing-all-features.ipynb). Learn which [algorithms are supported in ONNX](how-to-configure-auto-train.md#select-your-experiment-type).
+
+The ONNX runtime also supports C#, so you can use the model built automatically in your C# apps without any need for recoding or any of the network latencies that REST endpoints introduce. Learn more about [inferencing ONNX models with the ONNX runtime C# API](https://github.com/Microsoft/onnxruntime/blob/master/docs/CSharp_API.md). 
+
+## <a name="local-remote"></a>Guidance on local vs. remote managed ML compute targets
+
+The web interface for automated ML always uses a remote [compute target](concept-compute-target.md).  But when you use the Python SDK, you will choose either a local compute or a remote compute target for automated ML training.
+
+* **Local compute**: Training occurs on your local laptop or VM compute. 
+* **Remote compute**: Training occurs on Machine Learning compute clusters.  
 
-## The studio vs SDK
+### Choose compute target
+Consider these factors when choosing your compute target:
 
-Learn about the parity and differences between the high-level automated ML capabilities available through the Python SDK and the studio in Azure Machine Learning. 
+ * **Choose a local compute**: If your scenario is about initial explorations or demos using small data and short trains (i.e. seconds or a couple of minutes per child run), training on your local computer might be a better choice.  There is no setup time, the infrastructure resources (your PC or VM) are directly available.
+ * **Chose a remote ML compute cluster**: If you are training with larger datasets like in production training creating models which need longer trains, remote compute will provide much better end-to-end time performance because `AutoML` will parallelize trains across the cluster's nodes. On a remote compute, the start up time for the internal infrastructure will add around 1.5 minutes per child run, plus additional minutes for the cluster infrastructure if the VMs are not yet up and running.
+
+### Pros and cons
+Consider these pros and cons when choosing to use local vs. remote.
+
+|  | Pros (Advantages)  |Cons (Handicaps)  |
+|---------|---------|---------|---------|
+|**Local compute target** |  <li> No environment start up time   | <li>  Subset of features<li>  Can't parallelize runs <li> Worse for large data. <li>No data streaming while training <li>  No DNN-based featurization <li> Python SDK only |
+|**Remote ML compute clusters**|  <li> Full set of features <li> Parallelize child runs <li>   Large data support<li>  DNN-based featurization <li>  Dynamic scalability of compute cluster on demand <li> No-code experience (web UI) also available  |  <li> Start up time for cluster nodes <li> Start up time for each child run    |
+
+### Feature availability 
+
+ More features are available when you use the remote compute, as shown in the table below. Some of these features are available only in an Enterprise workspace.
+
+| Feature                                                    | Remote | Local | Requires <br>Enterprise workspace |
+|------------------------------------------------------------|--------|-------|-------------------------------|
+| Data streaming (Large data support, up to 100 GB)          | ✓      |       | ✓                             |
+| DNN-BERT-based text featurization and training             | ✓      |       | ✓                             |
+| Out-of-the-box GPU support (training and inference)        | ✓      |       | ✓                             |
+| Image Classification and Labeling support                  | ✓      |       | ✓                             |
+| Auto-ARIMA, Prophet and ForecastTCN models for forecasting | ✓      |       | ✓                             |
+| Multiple runs/iterations in parallel                       | ✓      |       | ✓                             |
+| Create models with interpretability in AutoML studio web experience UI      | ✓      |       | ✓                             |
+| Feature engineering customization in studio web experience UI                        | ✓      |       | ✓                              |
+| Azure ML hyperparameter tuning                             | ✓      |       |                               |
+| Azure ML Pipeline workflow support                         | ✓      |       |                               |
+| Continue a run                                             | ✓      |       |                               |
+| Forecasting                                                | ✓      | ✓     | ✓                             |
+| Create and run experiments in notebooks                    | ✓      | ✓     |                               |
+| Register and visualize experiment's info and metrics in UI | ✓      | ✓     |                               |
+| Data guardrails                                            | ✓      | ✓     |                               |
+
+
+## Automated ML in Azure Machine Learning
+
+Azure Machine Learning offers two experiences for working with automated ML
+
+* For code experienced customers, [Azure Machine Learning Python SDK](https://docs.microsoft.com/python/api/overview/azure/ml/intro?view=azure-ml-py) 
+
+* For limited/no code experience customers, Azure Machine Learning studio at [https://ml.azure.com](https://ml.azure.com/)  
+
+<a name="parity"></a>
 
 ### Experiment settings 
 
@@ -208,7 +298,9 @@ The ONNX runtime also supports C#, so you can use the model built automatically
 
 See examples and learn how to build models using automated machine learning:
 
-+ Follow the [Tutorial: Automatically train a regression model with Azure Machine Learning](tutorial-auto-train-models.md)
++ Follow the [Tutorial: Automatically train a regression model with Azure Machine Learning](tutorial-auto-train-models.md) 
+
++ Learn how to use a [remote compute target](how-to-auto-train-remote.md)
 
 + Configure the settings for automatic training experiment:
   + In Azure Machine Learning studio, [use these steps](how-to-use-automated-ml-for-ml-models.md).
diff --git a/articles/machine-learning/how-to-auto-train-remote.md b/articles/machine-learning/how-to-auto-train-remote.md
@@ -26,7 +26,9 @@ In this article, you learn how to build a model using automated ML with AmlCompu
 
 ## How does remote differ from local?
 
-The tutorial "[Train a classification model with automated machine learning](tutorial-auto-train-models.md)" teaches you how to use a local computer to train a model with automated ML. The workflow when training locally also applies to  remote targets as well. However, with remote compute, automated ML experiment iterations are executed asynchronously. This functionality allows you to cancel a particular iteration, watch the status of the execution, or continue to work on other cells in the Jupyter notebook. To train remotely, you first create a remote compute target such as AmlCompute. Then you configure the remote resource and submit your code there.
+More features are available when you use a remote compute target.  For more details, see [Local and remote compute targets](concept-automated-ml.md#local-remote).
+
+The tutorial "[Train a classification model with automated machine learning](tutorial-auto-train-models.md)" teaches you how to use a local computer to train a model with automated ML. The workflow when training locally also applies to  remote targets as well. To train remotely, you first create a remote compute target such as AmlCompute. Then you configure the remote resource and submit your code there.
 
 This article shows the extra steps needed to run an automated ML experiment on a remote AmlCompute target. The workspace object, `ws`, from the tutorial is used throughout the code here.
 
