initial commit

Author: EricWrightAtWork

articles/machine-learning/how-to-auto-train-forecast.md

@@ -294,7 +294,13 @@ best_run, fitted_model = local_run.get_output()
 
 ## Forecasting with best model
 
-Use the best model iteration to forecast values for data that wasn't used to train the model. 
+Use the best model iteration to forecast values for data that wasn't used to train the model.
+  
+### Evaluating model accuracy with a rolling forecast
+
+Before you put a model into production, you should evaluate its accuracy on a test set held out from the training data. A best practice procedure is a so-called rolling evaluation which rolls the trained forecaster forward in time over the test set, averaging error metrics over several prediction windows to obtain statistically robust estimates for some set of chosen metrics. Ideally, the test set for the evaluation is long relative to the model's forecast horizon; estimates of forecasting error may otherwise be statistically noisy and, therefore, less reliable.  
+    
+### Prediction into the future
 
 The [forecast_quantiles()](/python/api/azureml-train-automl-client/azureml.train.automl.model_proxy.modelproxy#forecast-quantiles-x-values--typing-any--y-values--typing-union-typing-any--nonetype----none--forecast-destination--typing-union-typing-any--nonetype----none--ignore-data-errors--bool---false-----azureml-data-abstract-dataset-abstractdataset) function allows specifications of when predictions should start, unlike the `predict()` method, which is typically used for classification and regression tasks. The forecast_quantiles() method by default generates a point forecast or a mean/median forecast which doesn't have a cone of uncertainty around it. Learn more in the [Forecasting away from training data notebook](https://github.com/Azure/azureml-examples/blob/main/v1/python-sdk/tutorials/automl-with-azureml/forecasting-forecast-function/auto-ml-forecasting-function.ipynb).