Merge pull request #116143 from garyericson/05-21-mi-ml-in-new-scheme

ShannonLeavitt · web-flow · commit c29fc9fa3f7d · 2020-05-27T10:44:52.000-06:00
ASQLMI: Adding ML to MI TOC
diff --git a/articles/azure-sql/managed-instance/bread/toc.yml b/articles/azure-sql/managed-instance/bread/toc.yml
@@ -0,0 +1,11 @@
+- name: Azure
+  tocHref: /sql/
+  topicHref: /azure/index
+  items:
+  - name: SQL Managed Instance
+    tocHref: /sql/
+    topicHref: /azure/azure-sql/managed-instance
+    items:
+    - name: Machine Learning
+      tocHref: /sql/machine-learning/
+      topicHref: /azure/azure-sql/managed-instance/machine-learning-services-overview
diff --git a/articles/azure-sql/managed-instance/context/ml-context.yml b/articles/azure-sql/managed-instance/context/ml-context.yml
@@ -0,0 +1,5 @@
+### YamlMime:ContextObject
+brand: azure
+uhfHeaderId: azure
+breadcrumb_path: ../bread/toc.yml
+toc_rel: ../../toc.yml
diff --git a/articles/azure-sql/managed-instance/machine-learning-services-differences.md b/articles/azure-sql/managed-instance/machine-learning-services-differences.md
@@ -0,0 +1,80 @@
+---
+title: Key differences for Machine Learning Services (preview)
+description: This topic describes key differences between Machine Learning Services in Azure SQL Managed Instance and SQL Server Machine Learning Services.
+services: sql-database
+ms.service: sql-database
+ms.subservice: machine-learning
+ms.custom: 
+ms.devlang: 
+ms.topic: conceptual
+author: garyericson
+ms.author: garye
+ms.reviewer: carlrab, davidph
+manager: cgronlun
+ms.date: 05/27/2020
+---
+
+# Key differences between Machine Learning Services in Azure SQL Managed Instance and SQL Server
+
+The functionality of [Machine Learning Services in Azure SQL Managed Instance (preview)](machine-learning-services-overview.md) is nearly identical to [SQL Server Machine Learning Services](https://docs.microsoft.com/sql/advanced-analytics/what-is-sql-server-machine-learning). Following are some key differences.
+
+> [!IMPORTANT]
+> Machine Learning Services in Azure SQL Managed Instance is currently in public preview. To sign up, see [Sign up for the preview](machine-learning-services-overview.md#signup).
+
+## Preview limitations
+
+During the preview, the service has the following limitations:
+
+- Loopback connections do not work (see [Loopback connection to SQL Server from a Python or R script](/sql/machine-learning/connect/loopback-connection)).
+- External Resource pools are not supported.
+- Only Python and R are supported. External languages such as Java cannot be added.
+- Scenarios using the [Message Passing Interface](https://docs.microsoft.com/message-passing-interface/microsoft-mpi) (MPI) are not supported.
+
+In case of a Service Level Objective (SLO) update, please update the SLO and raise a support ticket to re-enable the dedicated resource limits for R/Python.
+
+## Language support
+
+Machine Learning Services in SQL Managed Instance and SQL Server support both Python and R [extensibility framework](https://docs.microsoft.com/sql/advanced-analytics/concepts/extensibility-framework). The key differences are:
+
+- The initial versions of Python and R are different between Machine Learning Services in SQL Managed Instance and SQL Server:
+
+  |                      | Python | R     |
+  |----------------------|--------|-------|
+  | SQL Managed Instance | 3.7.1  | 3.5.2 |
+  | SQL Server           | 3.5.2  | 3.3.3 |
+
+- There is no need to configure `external scripts enabled` via `sp_configure`. Once you are [signed up](machine-learning-services-overview.md#signup) for the preview, machine learning is enabled for your SQL database.
+
+## Packages
+
+Python and R package management work differently between SQL Managed Instance and SQL Server. These differences are:
+
+- Packages cannot perform outbound network calls. This limitation is similar to the [default firewall rules for Machine Learning Services](https://docs.microsoft.com//sql/advanced-analytics/security/firewall-configuration) in SQL Server, but can't be changed in SQL Managed Instance.
+- There is no support for packages that depend on external runtimes (like Java) or need access to OS APIs for installation or usage.
+
+For more information about managing Python and R packages, see:
+
+- [Get Python package information](https://docs.microsoft.com/sql/machine-learning/package-management/python-package-information?context=azure/sql-database/context/ml-context&view=sql-server-ver15)
+- [Get R package information](https://docs.microsoft.com/sql/machine-learning/package-management/r-package-information?context=azure/sql-database/context/ml-context&view=sql-server-ver15)
+
+## Resource governance
+
+It is not possible to limit R resources through [Resource Governor](https://docs.microsoft.com/sql/relational-databases/resource-governor/resource-governor) and external resource pools.
+
+During the public preview, R resources are set to a maximum of 20% of the SQL Managed Instance resources, and depend on which service tier you choose. For more information, see [Azure SQL Database purchasing models](https://docs.microsoft.com/azure/sql-database/sql-database-service-tiers).
+
+### Insufficient memory error
+
+If there is insufficient memory available for R, you will get an error message. Common error messages are:
+
+- Unable to communicate with the runtime for 'R' script for request id: *******. Please check the requirements of 'R' runtime
+- 'R' script error occurred during execution of 'sp_execute_external_script' with HRESULT 0x80004004. ...an external script error occurred: "..could not allocate memory (0 Mb) in C function 'R_AllocStringBuffer'"
+- An external script error occurred: Error: cannot allocate vector of size.
+
+Memory usage depends on how much is used in your R scripts and the number of parallel queries being executed. If you receive the errors above, you can scale your database to a higher service tier to resolve this.
+
+## Next steps
+
+- See the overview, [Machine Learning Services in Azure SQL Managed Instance](machine-learning-services-overview.md).
+- To learn how to use Python in Machine Learning Services, see [Run Python scripts](https://docs.microsoft.com/sql/machine-learning/tutorials/quickstart-python-create-script?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15).
+- To learn how to use R in Machine Learning Services, see [Run R scripts](https://docs.microsoft.com/sql/machine-learning/tutorials/quickstart-r-create-script?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15).
diff --git a/articles/azure-sql/managed-instance/machine-learning-services-overview.md b/articles/azure-sql/managed-instance/machine-learning-services-overview.md
@@ -0,0 +1,60 @@
+---
+title: Machine Learning Services in Azure SQL Managed Instance (preview)
+description: This article provides an overview or Machine Learning Services in Azure SQL Managed Instance.
+services: sql-database
+ms.service: sql-database
+ms.subservice: machine-learning
+ms.custom: 
+ms.devlang: 
+ms.topic: conceptual
+author: garyericson
+ms.author: garye
+ms.reviewer: carlrab, davidph
+manager: cgronlun
+ms.date: 05/27/2020
+---
+
+# Machine Learning Services in Azure SQL Managed Instance (preview)
+
+Machine Learning Services is a feature of Azure SQL Managed Instance (preview) that provides in-database machine learning, supporting both Python and R scripts. The feature includes Microsoft Python and R packages for high-performance predictive analytics and machine learning. The relational data can be used in scripts through stored procedures, T-SQL script containing Python or R statements, or Python or R code containing T-SQL.
+
+> [!IMPORTANT]
+> Machine Learning Services is a feature of Azure SQL Managed Instance that's currently in public preview.
+> This preview functionality is initially available in a limited number of regions in the US, Asia Europe, and Australia with additional regions being added later.
+>
+> This preview version is provided without a service level agreement, and it's not recommended for production workloads. Certain features might not be supported or might have constrained capabilities.
+> For more information, see [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/).
+>
+> [Sign up for the preview](#signup) below.
+
+## What is Machine Learning Services?
+
+Machine Learning Services in Azure SQL Managed Instance lets you execute Python and R scripts in-database. You can use it to prepare and clean data, do feature engineering, and train, evaluate, and deploy machine learning models within a database. The feature runs your scripts where the data resides and eliminates transfer of the data across the network to another server.
+
+Use Machine Learning Services with R/Python support in Azure SQL Managed Instance to:
+
+- **Run R and Python scripts to do data preparation and general purpose data processing** - You can now bring your R/Python scripts to Azure SQL Managed Instance where your data lives, instead of having to move data out to some other server to run R and Python scripts. You can eliminate the need for data movement and associated problems related to latency, security, and compliance.
+
+- **Train machine learning models in database** - You can train models using any open source algorithms. You can easily scale your training to the entire dataset rather than relying on sample datasets pulled out of the database.
+
+- **Deploy your models and scripts into production in stored procedures** - The scripts and trained models can be operationalized simply by embedding them in T-SQL stored procedures. Apps connecting to Azure SQL Managed Instance can benefit from predictions and intelligence in these models by just calling a stored procedure. You can also use the native T-SQL PREDICT function to operationalize models for fast scoring in highly concurrent real-time scoring scenarios.
+
+Base distributions of Python and R are included in Machine Learning Services. You can install and use open-source packages and frameworks, such as PyTorch, TensorFlow, and scikit-learn, in addition to the Microsoft packages [revoscalepy](https://docs.microsoft.com/sql/advanced-analytics/python/ref-py-revoscalepy) and [microsoftml](https://docs.microsoft.com/sql/advanced-analytics/python/ref-py-microsoftml) for Python, and [RevoScaleR](https://docs.microsoft.com/sql/advanced-analytics/r/ref-r-revoscaler), [MicrosoftML](https://docs.microsoft.com/sql/advanced-analytics/r/ref-r-microsoftml), [olapR](https://docs.microsoft.com/sql/advanced-analytics/r/ref-r-olapr), and [sqlrutils](https://docs.microsoft.com/sql/advanced-analytics/r/ref-r-sqlrutils) for R.
+
+<a name="signup"></a>
+
+## Sign up for the preview
+
+This limited public preview is subject to the [Azure preview terms](https://azure.microsoft.com/support/legal/preview-supplemental-terms/). 
+
+If you're interested in joining the preview program and accept these terms, then you can request enrollment by creating an Azure support ticket at [https://azure.microsoft.com/support/create-ticket/](https://azure.microsoft.com/support/create-ticket/). For "Issue type" select "Technical", for "Service" select "SQL Database Managed Instance", and for "Problem type" select "Other". In your request, state that you would like to be enrolled into limited public preview of Machine Learning for SQL Managed Instance with these details: logical server name, region, and subscription ID.
+
+Once you're enrolled in the program, Microsoft will onboard you to the public preview and enable Machine Learning Services for your existing or new database.
+
+Machine Learning Services in SQL Managed Instance is not recommended for production workloads during the public preview.
+
+## Next steps
+
+- See the [key differences from SQL Server Machine Learning Services](machine-learning-services-differences.md).
+- To learn how to use Python in Machine Learning Services, see [Run Python scripts](https://docs.microsoft.com/sql/machine-learning/tutorials/quickstart-python-create-script?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15).
+- To learn how to use R in Machine Learning Services, see [Run R scripts](https://docs.microsoft.com/sql/machine-learning/tutorials/quickstart-r-create-script?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15).
diff --git a/articles/azure-sql/toc.yml b/articles/azure-sql/toc.yml
@@ -754,6 +754,101 @@
       href: managed-instance/connectivity-architecture-overview.md
     - name: T-SQL differences
       href: managed-instance/transact-sql-tsql-differences-sql-server.md
+
+    - name: Machine Learning Services
+      items:
+      - name: Overview
+        href: managed-instance/machine-learning-services-overview.md
+      - name: Key differences
+        href: managed-instance/machine-learning-services-differences.md
+      - name: Quickstarts
+        items:
+        - name: Python
+          items:
+          - name: Run Python scripts
+            href: https://docs.microsoft.com/sql/machine-learning/tutorials/quickstart-python-create-script?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+          - name: Data structures and objects
+            href: https://docs.microsoft.com/sql/machine-learning/tutorials/quickstart-python-data-structures?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+          - name: Python functions
+            href: https://docs.microsoft.com/sql/machine-learning/tutorials/quickstart-python-functions?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+          - name: Train and score a model
+            href: https://docs.microsoft.com/sql/machine-learning/tutorials/quickstart-python-train-score-model?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+        - name: R
+          items:
+          - name: Run R scripts
+            href: https://docs.microsoft.com/sql/machine-learning/tutorials/quickstart-r-create-script?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+          - name: Data types and objects
+            href: https://docs.microsoft.com/sql/machine-learning/tutorials/quickstart-r-data-types-and-objects?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+          - name: R functions
+            href: https://docs.microsoft.com/sql/machine-learning/tutorials/quickstart-r-functions?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+          - name: Train and score a model
+            href: https://docs.microsoft.com/sql/machine-learning/tutorials/quickstart-r-train-score-model?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+      - name: Tutorials
+        items:
+        - name: Python
+          items:
+          - name: Ski rental (linear regression)
+            items:
+            - name: 1 - Introduction
+              href: https://docs.microsoft.com/sql/machine-learning/tutorials/python-ski-rental-linear-regression?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+            - name: 2 - Prepare data
+              href: https://docs.microsoft.com/sql/machine-learning/tutorials/python-ski-rental-linear-regression-prepare-data?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+            - name: 3 - Train model
+              href: https://docs.microsoft.com/sql/machine-learning/tutorials/python-ski-rental-linear-regression-train-model?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+            - name: 4 - Deploy model
+              href: https://docs.microsoft.com/sql/machine-learning/tutorials/python-ski-rental-linear-regression-deploy-model?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+          - name: Categorize customers (k-means clustering)
+            items:
+            - name: 1 - Introduction
+              href: https://docs.microsoft.com/sql/machine-learning/tutorials/python-clustering-model?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+            - name: 2 - Prepare the data
+              href: https://docs.microsoft.com/sql/machine-learning/tutorials/python-clustering-model-prepare-data?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+            - name: 3 - Create the model
+              href: https://docs.microsoft.com/sql/machine-learning/tutorials/python-clustering-model-build?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+            - name: 4 - Deploy the model
+              href: https://docs.microsoft.com/sql/machine-learning/tutorials/python-clustering-model-deploy?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+        - name: R
+          items:
+          - name: Ski rental (decision tree)
+            items:
+            - name: 1 - Introduction
+              href: https://docs.microsoft.com/sql/machine-learning/tutorials/r-predictive-model-introduction?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+            - name: 2 - Prepare data
+              href: https://docs.microsoft.com/sql/machine-learning/tutorials/r-predictive-model-prepare-data?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+            - name: 3 - Train model
+              href: https://docs.microsoft.com/sql/machine-learning/tutorials/r-predictive-model-train?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+            - name: 4 - Deploy model
+              href: https://docs.microsoft.com/sql/machine-learning/tutorials/r-predictive-model-deploy?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+          - name: Categorize customers (k-means clustering)
+            items:
+            - name: 1 - Introduction
+              href: https://docs.microsoft.com/sql/machine-learning/tutorials/r-clustering-model-introduction?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+            - name: 2 - Prepare the data
+              href: https://docs.microsoft.com/sql/machine-learning/tutorials/r-clustering-model-prepare-data?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+            - name: 3 - Create the model
+              href: https://docs.microsoft.com/sql/machine-learning/tutorials/r-clustering-model-build?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+            - name: 4 - Deploy the model
+              href: https://docs.microsoft.com/sql/machine-learning/tutorials/r-clustering-model-deploy?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+      - name: Manage Python, R packages
+        items:
+        - name: Python
+          items:
+          - name: Get Python package information
+            href: https://docs.microsoft.com/sql/machine-learning/package-management/python-package-information?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+          - name: Install with sqlmlutils
+            href: https://docs.microsoft.com/sql/machine-learning/package-management/install-additional-python-packages-on-sql-server?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+        - name: R
+          items:
+          - name: Get R package information
+            href: https://docs.microsoft.com/sql/machine-learning/package-management/r-package-information?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+          - name: Install with sqlmlutils
+            href: https://docs.microsoft.com/sql/machine-learning/package-management/install-additional-r-packages-on-sql-server?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+          - name: Create a miniCRAN repo
+            href: https://docs.microsoft.com/sql/machine-learning/package-management/create-a-local-package-repository-using-minicran?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+          - name: Tips for using R packages
+            href: https://docs.microsoft.com/sql/machine-learning/package-management/tips-for-using-r-packages?context=/azure/azure-sql/managed-instance/context/ml-context&view=sql-server-ver15
+
+
     - name: Features
       items:
       - name: Linked servers
diff --git a/includes/sql-database-ml-preview-note.md b/includes/sql-database-ml-preview-note.md
