Merge pull request #66 from Rodrigossz/master

Fixing small issiues

Author: ruixinxu

… for Cosmos DB samples/MongoDB/README.md …r Cosmos DB samples/E-Commerce/README.mdNotebooks/PySpark/Synapse Link for Cosmos DB samples/MongoDB/README.md renamed to Notebooks/PySpark/Synapse Link for Cosmos DB samples/E-Commerce/README.md Notebooks/PySpark/Synapse Link for Cosmos DB samples/MongoDB/README.md renamed to Notebooks/PySpark/Synapse Link for Cosmos DB samples/E-Commerce/README.md

@@ -1,7 +1,7 @@
 
 # Load, Query, and Schema Updates with Azure Cosmos DB API for MongoDB
 
-In this noteboook, a simple dataset is created to show you how to use MongoDB client to ingest data, and how to use Synapse Link with Cosmos DB API for MongoDB to query this data. 
+In this notebook, a simple dataset is created to show you how to use MongoDB client to ingest data, and how to use Synapse Link with Cosmos DB API for MongoDB to query this data. 
 
 Also, we will ingest a second dataset with a schema update and show how it is managed by Synapse Link.
 

…/pyspark/01-CosmosDBSynapseMongoDB.ipynb …/pyspark/01-CosmosDBSynapseMongoDB.ipynbNotebooks/PySpark/Synapse Link for Cosmos DB samples/MongoDB/spark-notebooks/pyspark/01-CosmosDBSynapseMongoDB.ipynb renamed to Notebooks/PySpark/Synapse Link for Cosmos DB samples/E-Commerce/spark-notebooks/pyspark/01-CosmosDBSynapseMongoDB.ipynb Notebooks/PySpark/Synapse Link for Cosmos DB samples/MongoDB/spark-notebooks/pyspark/01-CosmosDBSynapseMongoDB.ipynb renamed to Notebooks/PySpark/Synapse Link for Cosmos DB samples/E-Commerce/spark-notebooks/pyspark/01-CosmosDBSynapseMongoDB.ipynb

@@ -18,6 +18,16 @@
       "source": [
         "# Getting started with Azure Cosmos DB's API for MongoDB and Synapse Link\n",
         "\n",
+        "## Key Information about this notebook\n",
+        "\n",
+        "* This notebook is part of the Azure Synapse Link for Azure Cosmos DB analitycal sample notebooks. For more information, click [here](../../../README.md). \n",
+        "\n",
+        "* It was build for Azure Cosmos DB API for MongoDB but you can, by yourself, customize it for Azure Cosmos DB SQL API. Please read about the analytical store inference schema differences between these 2 APIs [here](https://docs.microsoft.com/azure/cosmos-db/analytical-store-introduction#analytical-schema). \n",
+        "\n",
+        "* This is a Synapse Notebook and it was created to run in Synapse Analytics workspaces. Please make sure that you followed the pre-reqs of the [README](/README.md) file. After that, please execute the steps below in the same order that they are presented here. \n",
+        "\n",
+        "* From now on, all operations are case sentitive. Please be careful with everything you need to type.\n",
+        "\n",
         "In this sample we will execute the following tasks:\n",
         "\n",
         "1. Insert a dataset using the traditional MongoDB client.\n",
@@ -26,9 +36,9 @@
         "1. Execute aggregation queries again, consolidating both datasets.\n",
         "\n",
         "## Pre-requisites\n",
-        "1. Have you created a MongoDB API account in Azure Cosmos DB? If not, go to [Create an account for Azure Cosmos DB's API for MongoDB]().\n",
-        "1. For your Cosmos DB account, have you enabled Synapse Link? If not, go to [Enable Synapse Link for Azure Cosmos DB's API for MongoDB]().\n",
-        "1. Have you created a Synapse Workspace? If not, go to [Create Synapse Workspace account](). Please don't forget to add yourself as **Storage Blob Data Contributor** to the primary ADLS G2 account that is linked to the Synapse workspace.\n",
+        "1. Have you created a MongoDB API account in Azure Cosmos DB? If not, go to [Create an account for Azure Cosmos DB's API for MongoDB](https://docs.microsoft.com/azure/cosmos-db/mongodb-introduction).\n",
+        "1. For your Cosmos DB account, have you enabled Synapse Link? If not, go to [Enable Synapse Link for Azure Cosmos DB's API for MongoDB](https://docs.microsoft.com/azure/cosmos-db/configure-synapse-link).\n",
+        "1. Have you created a Synapse Workspace? If not, go to [Create Synapse Workspace account](https://docs.microsoft.com/azure/synapse-analytics/synapse-link/how-to-connect-synapse-link-cosmos-db). Please don't forget to add yourself as **Storage Blob Data Contributor** to the primary ADLS G2 account that is linked to the Synapse workspace.\n",
         "\n",
         "## Create a Cosmos DB collection with analytical store enabled\n",
         "\n",

…spark-notebooks/pyspark/requirements.txt …spark-notebooks/pyspark/requirements.txtNotebooks/PySpark/Synapse Link for Cosmos DB samples/MongoDB/spark-notebooks/pyspark/requirements.txt renamed to Notebooks/PySpark/Synapse Link for Cosmos DB samples/E-Commerce/spark-notebooks/pyspark/requirements.txt Notebooks/PySpark/Synapse Link for Cosmos DB samples/MongoDB/spark-notebooks/pyspark/requirements.txt renamed to Notebooks/PySpark/Synapse Link for Cosmos DB samples/E-Commerce/spark-notebooks/pyspark/requirements.txt

@@ -1,3 +1,3 @@
-pymongo==2.8.1
-aenum==2.2.4
+pymongo==2.8.1
+aenum==2.2.4
 bson==0.5.10

Notebooks/PySpark/Synapse Link for Cosmos DB samples/IoT/README.md

@@ -1,41 +1,44 @@
 
 # IoT Anomaly Detection leveraging Azure Synapse Link for Azure Cosmos DB
+
 The hypothetical scenario is Power Plant where signals from steam turbines are being analyzed and Anomalous signals are detected. You will ingest streaming and batch IoT data into Azure Cosmos DB using Azure Synapse Spark, perform Joins and aggregations using Azure Synapse Link and perform anomaly detection using Azure Cognitive Services on Spark (MMLSpark).
 
-### Environment setup
+## Environment setup
+
 Please make sure that you followed the pre-reqs of the main [README](../README.md) file. Please execute the below steps in the given order.
-1. Using the Data / Linked tab of your Synapse workspace, create IoTData folder within the root directory of the storage account that is attached to the Synapse workspace. Upload to this folder the "IoTDeviceInfo.csv" file that is placed under the "IoTData" dir of this repo. 
- 
+
+1. Using the Data / Linked tab of your Synapse workspace, create IoTData folder within the root directory of the storage account that is attached to the Synapse workspace. Upload to this folder the "IoTDeviceInfo.csv" file that is placed under the "IoTData" dir of this repo.
+
 ![upload_datasets](images/upload_datasets.png)
 
-2.  Using the Azure Portal, go to the Access Control (IAM) tab of the storage account associated with Synapse workspace, click on the +Add and Add a role assignment and add yourself to the Data Contributor role. This is needed for any spark metadata operations such as creating databases and tables using the Azure Synapse Spark Pool.
+2. Using the Azure Portal, go to the Access Control (IAM) tab of the storage account associated with Synapse workspace, click on the +Add and Add a role assignment and add yourself to the Data Contributor role. This is needed for any spark metadata operations such as creating databases and tables using the Azure Synapse Spark Pool.
 
-3. Using the Azure Portal, go to Data Explorer of your the Azure Cosmos DB Account and create a database called CosmosDBIoTDemo. 
+3. Using the Azure Portal, go to Data Explorer of your the Azure Cosmos DB Account and create a database called CosmosDBIoTDemo.
 
-4. In the same Data Explorer, create two Analytical Store enabled containers: IoTSignals and IoTDeviceInfo. In the portal interface, the container-id is the container name. Change the Throughput to Autoscale and set the max limit to 4,000. Please click [here](https://review.docs.microsoft.com/en-us/azure/cosmos-db/configure-synapse-link?branch=release-build-cosmosdb#create-analytical-ttl) for details on how to enable Analytical storage on Cosmos DB containers.
-    * Use /id as the Partition key for both the containers
+4. In the same Data Explorer, create two Analytical Store enabled containers: IoTSignals and IoTDeviceInfo. In the portal interface, the container-id is the container name. Change the Throughput to `autoscale` and set the max limit to 4,000. Please click [here](https://docs.microsoft.com/azure/cosmos-db/configure-synapse-link#azure-portal-2) for details on how to enable Analytical storage on Cosmos DB containers.
+    * Use `/id` as the Partition key for both the containers
     * Please make sure that Analytical store is enabled for both the containers
 
-5. In your Azure Synapse workspace, go to the Manage / Linked Services tab and create a linked service called CosmosDBIoTDemo pointing to the Cosmos DB database that was created in step 3 above. Please click [here](https://review.docs.microsoft.com/en-us/azure/synapse-analytics/synapse-link/how-to-connect-synapse-link-cosmos-db?branch=release-build-synapse#connect-an-azure-cosmos-db-database-to-a-synapse-workspace) for more details on creating Synapse linked service pointing to Cosmos DB.
+5. In your Azure Synapse workspace, go to the Manage / Linked Services tab and create a linked service called CosmosDBIoTDemo pointing to the Cosmos DB database that was created in step 3 above. Please click [here](https://docs.microsoft.com/azure/synapse-analytics/synapse-link/how-to-connect-synapse-link-cosmos-db) for more details on creating Synapse linked service pointing to Cosmos DB.
+
+## Notebooks Execution
 
-### Notebooks Execution
+Import the below four synapse spark notebooks under the `IoT/spark-notebooks/pyspark/` dir on to the Synapse workspace and attach the Spark pool created in the prerequisite to the notebooks.
 
-Import the below four synapse spark notebooks under the "IoT/spark-notebooks/pyspark/" dir on to the Synapse workspace and attach the Spark pool created in the prerequisite to the notebooks.
-1. [01-CosmosDBSynapseStreamIngestion: Ingest streaming data into Azure Cosmos DB collection using Structured Streaming](IoT/spark-notebooks/pyspark/01-CosmosDBSynapseStreamIngestion.ipynb)
+1. [01-CosmosDBSynapseStreamIngestion: Ingest streaming data into Azure Cosmos DB collection using Structured Streaming](./spark-notebooks/pyspark/01-CosmosDBSynapseStreamIngestion.ipynb)
 
     This notebook ingests documents to "IoTSignals" collection using structured streaming. Please make sure to stop the execution of this notebook after few 2 to 5 minutes of streaming, which would bring in enough documents required for the Anomaly detection in "04-CosmosDBSynapseML" notebook.
     Once the notebook execution is stopped, go to the Data Explorer in Azure Cosmos DB Account portal and make sure that the data has been loaded into the "IoTSignals" collection.   
 
-1. [02-CosmosDBSynapseBatchIngestion: Ingest Batch data into Azure Cosmos DB collection using Azure Synapse Spark](IoT/spark-notebooks/pyspark/02-CosmosDBSynapseBatchIngestion.ipynb)
+1. [02-CosmosDBSynapseBatchIngestion: Ingest Batch data into Azure Cosmos DB collection using Azure Synapse Spark](./spark-notebooks/pyspark/02-CosmosDBSynapseBatchIngestion.ipynb)
 
     This notebook ingests documents from "IoTDeviceInfo.csv" to the "IoTDeviceInfo" collection.
     Once the notebook execution is completed, go to the Data Explorer in Azure Cosmos DB Account portal and make sure that the data has been loaded into the "IoTDeviceInfo" collection.   
 
-1. [03-CosmosDBSynapseJoins: Perform Joins and aggregations across Azure Cosmos DB collections using Azure Synapse Link](IoT/spark-notebooks/pyspark/03-CosmosDBSynapseJoins.ipynb)
+1. [03-CosmosDBSynapseJoins: Perform Joins and aggregations across Azure Cosmos DB collections using Azure Synapse Link](./spark-notebooks/pyspark/03-CosmosDBSynapseJoins.ipynb)
 
     This notebook creates Spark tables pointing to Azure Cosmos DB Analytical store collections, perform Joins, filters and aggregations across collectionsand visualize the data using plotly
 
-1. [04-CosmosDBSynapseML: Perform Anomaly Detection using Azure Synapse Link and Azure Cognitive Services on Synapse Spark (MMLSpark)](IoT/spark-notebooks/pyspark/04-CosmosDBSynapseML.ipynb)
-
-    This notebook performs anomaly detection using Azure Cognitive Services on Spark and enables to visualize the anomalies using plotly.
+1. [04-CosmosDBSynapseML: Perform Anomaly Detection using Azure Synapse Link and Azure Cognitive Services on Synapse Spark (MMLSpark)](./spark-notebooks/pyspark/04-CosmosDBSynapseML.ipynb)
 
+    This notebook performs anomaly detection using Azure Cognitive Services on Spark and enables to visualize the anomalies using `plotly`.

Notebooks/PySpark/Synapse Link for Cosmos DB samples/IoT/spark-notebooks/pyspark/01-CosmosDBSynapseStreamIngestion.ipynb

@@ -2,7 +2,17 @@
   "metadata": {
     "saveOutput": false,
     "language_info": {
-      "name": "python"
+      "name": "python",
+      "version": "3.8.5-final"
+    },
+    "kernelspec": {
+      "name": "python3",
+      "display_name": "Python 3.8.5 32-bit",
+      "metadata": {
+        "interpreter": {
+          "hash": "bb162378bcd0865be49969664200f47400a735e3f46a0ab8d5d80e6092c9afde"
+        }
+      }
     }
   },
   "nbformat": 4,
@@ -14,18 +24,28 @@
       "source": [
         "# Streaming ingestion into Azure Cosmos DB collection using Structured Streaming\n",
         "\n",
-        "In this notebook, we'll \n",
+        "## Key Information about this notebook\n",
+        "\n",
+        "* This notebook is part of the Azure Synapse Link for Azure Cosmos DB analitycal sample notebooks. For more information, click [here](../../../README.md). \n",
+        "\n",
+        "* It was build for Azure Cosmos DB SQL API but you can, by yourself, customize it for Azure Cosmos DB API for MongoDB. Please read about the analytical store inference schema differences between these 2 APIs [here](https://docs.microsoft.com/azure/cosmos-db/analytical-store-introduction#analytical-schema). \n",
+        "\n",
+        "* This is a Synapse Notebook and it was created to run in Synapse Analytics workspaces. Please make sure that you followed the pre-reqs of the [README](/README.md) file. After that, please execute the steps below in the same order that they are presented here. \n",
+        "\n",
+        "* From now on, all operations are case sentitive. Please be careful with everything you need to type.\n",
+        "\n",
+        "In this notebook, we'll:\n",
         "\n",
         "1. Simulate streaming data generation using Rate streaming source\n",
         "2. Format the stream dataframe as per the IoTSignals schema\n",
         "3. Write the streaming dataframe to the Azure Cosmos DB collection\n",
         "\n",
-        ">**Did you know?** Azure Cosmos DB is a great fit for IoT predictive maintenance and anomaly detection use cases. [Click here](https://review.docs.microsoft.com/en-us/azure/cosmos-db/synapse-link-use-cases?branch=release-build-cosmosdb#iot-predictive-maintenance) to learn more about an IoT architecture leveraging HTAP capabilities of Azure Synapse Link for Azure Cosmos DB.\n",
+        ">**Did you know?** Azure Cosmos DB is a great fit for IoT predictive maintenance and anomaly detection use cases. [Click here](https://docs.microsoft.com/azure/cosmos-db/) to learn more about an IoT architecture leveraging HTAP capabilities of Azure Synapse Link for Azure Cosmos DB.\n",
         "\n",
-        ">**Did you know?**  [Azure Synapse Link for Azure Cosmos DB](https://review.docs.microsoft.com/en-us/azure/cosmos-db/synapse-link?branch=release-build-cosmosdb) is a hybrid transactional and analytical processing (HTAP) capability that enables you to run near real-time analytics over operational data in Azure Cosmos DB.\n",
+        ">**Did you know?**  [Azure Synapse Link for Azure Cosmos DB](https://docs.microsoft.com/azure/cosmos-db/synapse-link) is a hybrid transactional and analytical processing (HTAP) capability that enables you to run near real-time analytics over operational data in Azure Cosmos DB.\n",
         " \n",
         "\n",
-        ">**Did you know?**  [Azure Cosmos DB analytical store](https://review.docs.microsoft.com/en-us/azure/cosmos-db/analytical-store-introduction?branch=release-build-cosmosdb) is a fully isolated column store for enabling large scale analytics against operational data in your Azure Cosmos DB, without any impact to your transactional workloads.\n",
+        ">**Did you know?**  [Azure Cosmos DB analytical store](https://docs.microsoft.com/azure/cosmos-db/analytical-store-introduction) is a fully isolated column store for enabling large scale analytics against operational data in your Azure Cosmos DB, without any impact to your transactional workloads.\n",
         " "
       ],
       "attachments": {}

Notebooks/PySpark/Synapse Link for Cosmos DB samples/IoT/spark-notebooks/pyspark/02-CosmosDBSynapseBatchIngestion.ipynb

@@ -14,7 +14,17 @@
       "source": [
         "# Batch ingestion into Azure Cosmos DB collection\n",
         "\n",
-        "In this notebook, we'll \n",
+        "## Key Information about this notebook\n",
+        "\n",
+        "* This notebook is part of the Azure Synapse Link for Azure Cosmos DB analitycal sample notebooks. For more information, click [here](../../../README.md). \n",
+        "\n",
+        "* It was build for Azure Cosmos DB SQL API but you can, by yourself, customize it for Azure Cosmos DB API for MongoDB. Please read about the analytical store inference schema differences between these 2 APIs [here](https://docs.microsoft.com/azure/cosmos-db/analytical-store-introduction#analytical-schema). \n",
+        "\n",
+        "* This is a Synapse Notebook and it was created to run in Synapse Analytics workspaces. Please make sure that you followed the pre-reqs of the [README](/README.md) file. After that, please execute the steps below in the same order that they are presented here. \n",
+        "\n",
+        "* From now on, all operations are case sentitive. Please be careful with everything you need to type.\n",
+        "\n",
+        "In this notebook, we'll:\n",
         "\n",
         "1. Load the IoTDeviceInfo dataset from ADLS Gen2 to a dataframe\n",
         "2. Write the dataframe to the Azure Cosmos DB collection\n",
@@ -23,8 +33,7 @@
         " \n",
         "\n",
         ">**Did you know?**  [Azure Cosmos DB analytical store](https://review.docs.microsoft.com/en-us/azure/cosmos-db/analytical-store-introduction?branch=release-build-cosmosdb) is a fully isolated column store for enabling large scale analytics against operational data in your Azure Cosmos DB, without any impact to your transactional workloads.\n",
-        " \n",
-        ""
+        " \n"
       ],
       "attachments": {}
     },
@@ -42,7 +51,7 @@
     },
     {
       "cell_type": "code",
-      "execution_count": 3,
+      "execution_count": null,
       "outputs": [],
       "metadata": {},
       "source": [
@@ -66,7 +75,7 @@
     },
     {
       "cell_type": "code",
-      "execution_count": 4,
+      "execution_count": null,
       "outputs": [],
       "metadata": {},
       "source": [
@@ -82,4 +91,4 @@
       "attachments": {}
     }
   ]
-}
+}