lower-case big data, updates for synapseml, formatting fixes

Author: HeidiSteen

articles/cognitive-services/TOC.yml

@@ -87,7 +87,7 @@
       href: ./security-controls-policy.md
   - name: Use with big data
     items:
-    - name: Cognitive Services for Big Data
+    - name: Cognitive Services for big data
       href: ./big-data/cognitive-services-for-big-data.md
     - name: Getting started
       href: ./big-data/getting-started.md

articles/cognitive-services/big-data/cognitive-services-for-big-data.md

@@ -1,6 +1,6 @@
 ---
-title: "Cognitive Services for Big Data"
-description: Learn how to leverage Azure Cognitive Services on large datasets using Python, Java, and Scala. With Cognitive Services for Big Data you can embed continuously improving, intelligent models directly into Apache Spark™ and SQL computations.
+title: "Cognitive Services for big data"
+description: Learn how to leverage Azure Cognitive Services on large datasets using Python, Java, and Scala. With Cognitive Services for big data you can embed continuously improving, intelligent models directly into Apache Spark™ and SQL computations.
 services: cognitive-services
 author: mhamilton723
 manager: nitinme
@@ -10,17 +10,17 @@ ms.date: 10/28/2021
 ms.author: marhamil
 ---
 
-# Azure Cognitive Services for Big Data
+# Azure Cognitive Services for big data
 
-![Azure Cognitive Services for Big Data](media/cognitive-services-big-data-overview.svg)
+![Azure Cognitive Services for big data](media/cognitive-services-big-data-overview.svg)
 
-The Azure Cognitive Services for Big Data lets users channel terabytes of data through Cognitive Services using [Apache Spark™](/dotnet/spark/what-is-spark). With the Cognitive Services for Big Data, it's easy to create large-scale intelligent applications with any datastore.
+The Azure Cognitive Services for big data lets users channel terabytes of data through Cognitive Services using [Apache Spark™](/dotnet/spark/what-is-spark). With the Cognitive Services for big data, it's easy to create large-scale intelligent applications with any datastore.
 
-With Cognitive Services for Big Data you can embed continuously improving, intelligent models directly into Apache Spark™ and SQL computations. These tools liberate developers from low-level networking details, so that they can focus on creating smart, distributed applications.
+With Cognitive Services for big data you can embed continuously improving, intelligent models directly into Apache Spark™ and SQL computations. These tools liberate developers from low-level networking details, so that they can focus on creating smart, distributed applications.
 
 ## Features and benefits
 
-Cognitive Services for Big Data can use services from any region in the world, as well as [containerized Cognitive Services](../cognitive-services-container-support.md). Containers support low or no connectivity deployments with ultra-low latency responses. Containerized Cognitive Services can be run locally, directly on the worker nodes of your Spark cluster, or on an external orchestrator like Kubernetes.
+Cognitive Services for big data can use services from any region in the world, as well as [containerized Cognitive Services](../cognitive-services-container-support.md). Containers support low or no connectivity deployments with ultra-low latency responses. Containerized Cognitive Services can be run locally, directly on the worker nodes of your Spark cluster, or on an external orchestrator like Kubernetes.
 
 ## Supported services
 
@@ -57,9 +57,9 @@ Cognitive Services for Big Data can use services from any region in the world, a
 |:-----------|:------------------|
 |[Bing Image Search](/azure/cognitive-services/bing-image-search "Bing Image Search")|The Bing Image Search service returns a display of images determined to be relevant to the user's query.|
 
-## Supported programming languages for Cognitive Services for Big Data
+## Supported programming languages for Cognitive Services for big data
 
-The Cognitive Services for Big Data are built on Apache Spark. Apache Spark is a distributed computing library that supports Java, Scala, Python, R, and many other languages. These languages are currently supported.
+The Cognitive Services for big data are built on Apache Spark. Apache Spark is a distributed computing library that supports Java, Scala, Python, R, and many other languages. These languages are currently supported.
 
 ### Python
 
@@ -71,7 +71,7 @@ We provide a Scala and Java-based Spark API in the `com.microsoft.ml.spark.cogni
 
 ## Supported platforms and connectors
 
-The Cognitive Services for Big Data requires Apache Spark. There are several Apache Spark platforms that support the Cognitive Services for Big Data.
+The Cognitive Services for big data requires Apache Spark. There are several Apache Spark platforms that support the Cognitive Services for big data.
 
 ### Azure Databricks
 
@@ -100,15 +100,15 @@ The basis of Spark is the DataFrame: a tabular collection of data distributed ac
    - Do SQL-style computations such as join and filter tables.
    - Apply functions to large datasets using MapReduce style parallelism.
    - Apply Distributed Machine Learning using Microsoft Machine Learning for Apache Spark.
-   - Use the Cognitive Services for Big Data to enrich your data with ready-to-use intelligent services.
+   - Use the Cognitive Services for big data to enrich your data with ready-to-use intelligent services.
 
 ### Microsoft Machine Learning for Apache Spark (MMLSpark)
 
-[Microsoft Machine Learning for Apache Spark](https://mmlspark.blob.core.windows.net/website/index.html#install) (MMLSpark) is an open-source, distributed machine learning library (ML) built on Apache Spark. The Cognitive Services for Big Data is included in this package. Additionally, MMLSpark contains several other ML tools for Apache Spark, such as LightGBM, Vowpal Wabbit, OpenCV, LIME, and more. With MMLSpark, you can build powerful predictive and analytical models from any Spark datasource.
+[Microsoft Machine Learning for Apache Spark](https://mmlspark.blob.core.windows.net/website/index.html#install) (MMLSpark) is an open-source, distributed machine learning library (ML) built on Apache Spark. The Cognitive Services for big data is included in this package. Additionally, MMLSpark contains several other ML tools for Apache Spark, such as LightGBM, Vowpal Wabbit, OpenCV, LIME, and more. With MMLSpark, you can build powerful predictive and analytical models from any Spark datasource.
 
 ### HTTP on Spark
 
-Cognitive Services for Big Data is an example of how we can integrate intelligent web services with big data. Web services power many applications across the globe and most services communicate through the Hypertext Transfer Protocol (HTTP). To work with *arbitrary* web services at large scales, we provide HTTP on Spark. With HTTP on Spark, you can pass terabytes of data through any web service. Under the hood, we use this technology to power Cognitive Services for Big Data.
+Cognitive Services for big data is an example of how we can integrate intelligent web services with big data. Web services power many applications across the globe and most services communicate through the Hypertext Transfer Protocol (HTTP). To work with *arbitrary* web services at large scales, we provide HTTP on Spark. With HTTP on Spark, you can pass terabytes of data through any web service. Under the hood, we use this technology to power Cognitive Services for big data.
 
 ## Developer samples
 
@@ -126,11 +126,11 @@ Cognitive Services for Big Data is an example of how we can integrate intelligen
 
 - [The Azure Cognitive Services on Spark: Clusters with Embedded Intelligent Services](https://databricks.com/session/the-azure-cognitive-services-on-spark-clusters-with-embedded-intelligent-services)
 - [Spark Summit Keynote: Scalable AI for Good](https://databricks.com/session_eu19/scalable-ai-for-good)
-- [The Cognitive Services for Big Data in Cosmos DB](https://medius.studios.ms/Embed/Video-nc/B19-BRK3004?latestplayer=true&l=2571.208093)
+- [The Cognitive Services for big data in Cosmos DB](https://medius.studios.ms/Embed/Video-nc/B19-BRK3004?latestplayer=true&l=2571.208093)
 - [Lightning Talk on Large Scale Intelligent Microservices](https://www.youtube.com/watch?v=BtuhmdIy9Fk&t=6s)
 
 ## Next steps
 
-- [Getting Started with the Cognitive Services for Big Data](getting-started.md)
+- [Getting Started with the Cognitive Services for big data](getting-started.md)
 - [Simple Python Examples](samples-python.md)
 - [Simple Scala Examples](samples-scala.md)

articles/cognitive-services/big-data/getting-started.md

@@ -1,12 +1,12 @@
 ---
-title: "Get started with Cognitive Services for Big Data"
-description: Set up your MMLSpark pipeline with Cognitive Services in Azure Databricks and run a sample.
+title: "Get started with Cognitive Services for big data"
+description: Set up your SynapseML or MMLSpark pipeline with Cognitive Services in Azure Databricks and run a sample.
 services: cognitive-services
 author: mhamilton723
 manager: nitinme
 ms.service: cognitive-services
 ms.topic: how-to
-ms.date: 10/28/2021
+ms.date: 08/16/2022
 ms.author: marhamil
 ms.devlang: python
 ms.custom: mode-other
@@ -16,15 +16,16 @@ ms.custom: mode-other
 
 Setting up your environment is the first step to building a pipeline for your data. After your environment is ready, running a sample is quick and easy.
 
-In this article, we'll perform these steps to get you started:
+In this article, you'll perform these steps to get started:
 
-1. [Create a Cognitive Services resource](#create-a-cognitive-services-resource)
-1. [Create an Apache Spark Cluster](#create-an-apache-spark-cluster)
-1. [Try a sample](#try-a-sample)
+> [!div class="checklist"]
+> * [Create a Cognitive Services resource](#create-a-cognitive-services-resource)
+> * [Create an Apache Spark cluster](#create-an-apache-spark-cluster)
+> * [Try a sample](#try-a-sample)
 
 ## Create a Cognitive Services resource
 
-To use the Big Data Cognitive Services, you must first create a Cognitive Service for your workflow. There are two main types of Cognitive Services: cloud services hosted in Azure and containerized services managed by users. We recommend starting with the simpler cloud-based Cognitive Services.
+To work with big data in Cognitive Services, first create a Cognitive Services resource for your workflow. There are two main types of Cognitive Services: cloud services hosted in Azure and containerized services managed by users. We recommend starting with the simpler cloud-based Cognitive Services.
 
 ### Cloud services
 
@@ -46,21 +47,30 @@ Follow [this guide](../cognitive-services-container-support.md?tabs=luis) to cre
 
 ## Create an Apache Spark cluster
 
-[Apache Spark™](http://spark.apache.org/) is a distributed computing framework designed for big-data data processing. Users can work with Apache Spark in Azure with services like Azure Databricks, Azure Synapse Analytics, HDInsight, and Azure Kubernetes Services. To use the Big Data Cognitive Services, you must first create a cluster. If you already have a Spark cluster, feel free to try an example.
+[Apache Spark™](http://spark.apache.org/) is a distributed computing framework designed for big-data data processing. Users can work with Apache Spark in Azure with services like Azure Databricks, Azure Synapse Analytics, HDInsight, and Azure Kubernetes Services. To use the big data Cognitive Services, you must first create a cluster. If you already have a Spark cluster, feel free to try an example.
 
 ### Azure Databricks
 
-Azure Databricks is an Apache Spark-based analytics platform with a one-click setup, streamlined workflows, and an interactive workspace. It's often used to collaborate between data scientists, engineers, and business analysts. To use the Big Data Cognitive Services on Azure Databricks, follow these steps:
+Azure Databricks is an Apache Spark-based analytics platform with a one-click setup, streamlined workflows, and an interactive workspace. It's often used to collaborate between data scientists, engineers, and business analysts. To use the big data Cognitive Services on Azure Databricks, follow these steps:
 
 1. [Create an Azure Databricks workspace](/azure/databricks/scenarios/quickstart-create-databricks-workspace-portal#create-an-azure-databricks-workspace)
+
 1. [Create a Spark cluster in Databricks](/azure/databricks/scenarios/quickstart-create-databricks-workspace-portal#create-a-spark-cluster-in-databricks)
-1. Install the Big Data Cognitive Services
+
+1. Install the SynapseML open-source library (or MMLSpark library if you're supporting a legacy application):
+
     * Create a new library in your databricks workspace  
        <img src="media/create-library.png" alt="Create library" width="50%"/>
-    * Input the following maven coordinates
+
+    * For SynapseML: input the following maven coordinates
+      Coordinates:  `com.microsoft.azure:synapseml_2.12:0.10.0`
+      Repository: default
+
+    * For MMLSpark (legacy): input the following maven coordinates
       Coordinates:  `com.microsoft.ml.spark:mmlspark_2.11:1.0.0-rc3`
       Repository: `https://mmlspark.azureedge.net/maven`  
       <img src="media/library-coordinates.png" alt="Library Coordinates" width="50%"/>
+
     * Install the library onto a cluster  
       <img src="media/install-library.png" alt="Install Library on Cluster" width="50%"/>
 
@@ -69,9 +79,10 @@ Azure Databricks is an Apache Spark-based analytics platform with a one-click se
 Optionally, you can use Synapse Analytics to create a spark cluster. Azure Synapse Analytics brings together enterprise data warehousing and big data analytics. It gives you the freedom to query data on your terms, using either serverless on-demand or provisioned resources at scale. To get started using Azure Synapse Analytics, follow these steps:
 
 1. [Create a Synapse Workspace (preview)](../../synapse-analytics/quickstart-create-workspace.md).
+
 1. [Create a new serverless Apache Spark pool (preview) using the Azure portal](../../synapse-analytics/quickstart-create-apache-spark-pool-portal.md).
 
-In Azure Synapse Analytics, Big Data for Cognitive Services is installed by default.
+In Azure Synapse Analytics, big data for Cognitive Services is installed by default.
 
 ### Azure Kubernetes Service
 
@@ -80,12 +91,14 @@ If you're using containerized Cognitive Services, one popular option for deployi
 To get started on Azure Kubernetes Service, follow these steps:
 
 1. [Deploy an Azure Kubernetes Service (AKS) cluster using the Azure portal](../../aks/learn/quick-kubernetes-deploy-portal.md)
+
 1. [Install the Apache Spark 2.4.0 helm chart](https://hub.helm.sh/charts/microsoft/spark)
+
 1. [Install a cognitive service container using Helm](../computer-vision/deploy-computer-vision-on-premises.md)
 
 ## Try a sample
 
-After you set up your Spark cluster and environment, you can run a short sample. This section demonstrates how to use the Big Data for Cognitive Services in Azure Databricks.
+After you set up your Spark cluster and environment, you can run a short sample. This sample assumes Azure Databricks and the `mmlspark.cognitive` package.
 
 First, you can create a notebook in Azure Databricks. For other Spark cluster providers, use their notebooks or Spark Submit.
 
@@ -101,36 +114,39 @@ First, you can create a notebook in Azure Databricks. For other Spark cluster pr
 
 1. Paste this code snippet into your new notebook.
 
-```python
-from mmlspark.cognitive import *
-from pyspark.sql.functions import col
-
-# Add your subscription key from the Language service (or a general Cognitive Service key)
-service_key = "ADD-SUBSCRIPTION-KEY-HERE"
-
-df = spark.createDataFrame([
-  ("I am so happy today, its sunny!", "en-US"),
-  ("I am frustrated by this rush hour traffic", "en-US"),
-  ("The cognitive services on spark aint bad", "en-US"),
-], ["text", "language"])
-
-sentiment = (TextSentiment()
-    .setTextCol("text")
-    .setLocation("eastus")
-    .setSubscriptionKey(service_key)
-    .setOutputCol("sentiment")
-    .setErrorCol("error")
-    .setLanguageCol("language"))
-
-results = sentiment.transform(df)
-
-# Show the results in a table
-display(results.select("text", col("sentiment")[0].getItem("score").alias("sentiment")))
-
-```
+  ```python
+  from mmlspark.cognitive import *
+  from pyspark.sql.functions import col
+  
+  # Add your region and subscription key from the Language service (or a general Cognitive Service key)
+  # If using a multi-region Cognitive Services resource, delete the placeholder text: service_region = ""
+  service_key = "ADD-SUBSCRIPTION-KEY-HERE"
+  service_region = "ADD-SERVICE-REGION-HERE"
+  
+  df = spark.createDataFrame([
+    ("I am so happy today, its sunny!", "en-US"),
+    ("I am frustrated by this rush hour traffic", "en-US"),
+    ("The cognitive services on spark aint bad", "en-US"),
+  ], ["text", "language"])
+  
+  sentiment = (TextSentiment()
+      .setTextCol("text")
+      .setLocation(service_region)
+      .setSubscriptionKey(service_key)
+      .setOutputCol("sentiment")
+      .setErrorCol("error")
+      .setLanguageCol("language"))
+  
+  results = sentiment.transform(df)
+  
+  # Show the results in a table
+  display(results.select("text", col("sentiment")[0].getItem("score").alias("sentiment")))
+  ```
 
 1. Get your subscription key from the **Keys and Endpoint** menu from your Language resource in the Azure portal.
+
 1. Replace the subscription key placeholder in your Databricks notebook code with your subscription key.
+
 1. Select the play, or triangle, symbol in the upper right of your notebook cell to run the sample. Optionally, select **Run All** at the top of your notebook to run all cells. The answers will display below the cell in a table.
 
 ### Expected results

articles/cognitive-services/big-data/recipes/anomaly-detection.md

@@ -1,7 +1,7 @@
 ---
-title: "Recipe: Predictive maintenance with the Cognitive Services for Big Data"
+title: "Recipe: Predictive maintenance with the Cognitive Services for big data"
 titleSuffix: Azure Cognitive Services
-description: This quickstart shows how to perform distributed anomaly detection with the Cognitive Services for Big Data
+description: This quickstart shows how to perform distributed anomaly detection with the Cognitive Services for big data
 services: cognitive-services
 author: mhamilton723
 manager: nitinme
@@ -14,7 +14,7 @@ ms.devlang: python
 ms.custom: devx-track-python
 ---
 
-# Recipe: Predictive maintenance with the Cognitive Services for Big Data
+# Recipe: Predictive maintenance with the Cognitive Services for big data
 
 This recipe shows how you can use Azure Synapse Analytics and Cognitive Services on Apache Spark for predictive maintenance of IoT devices. We'll follow along with the [CosmosDB and Synapse Link](https://github.com/Azure-Samples/cosmosdb-synapse-link-samples) sample. To keep things simple, in this recipe we'll read the data straight from a CSV file rather than getting streamed data through CosmosDB and Synapse Link. We strongly encourage you to look over the Synapse Link sample.