Merge pull request #52919 from MicrosoftDocs/release-ignite-kusto

Ignite Merge Down

Author: Cory Fowler

articles/data-explorer/check-cluster-health.md

@@ -0,0 +1,35 @@
+---
+title: Check the health of an Azure Data Explorer cluster
+description: This article describes steps to determine if your Azure Data Explorer cluster is healthy.
+author: orspod
+ms.author: v-orspod
+ms.reviewer: mblythe
+ms.service: data-explorer
+services: data-explorer
+ms.topic: conceptual
+ms.date: 09/24/2018
+---
+
+# Check the health of an Azure Data Explorer cluster
+
+There are several factors that impact the health of an Azure Data Explorer cluster, including CPU, memory, and the disk subsystem. This article shows some basic steps you can take to gauge the health of a cluster.
+
+1. Sign in to [https://dataexplorer.azure.com](https://dataexplorer.azure.com).
+
+1. In the left pane, select your cluster, and run the following command.
+
+    ```Kusto
+    .show diagnostics
+    | project IsHealthy
+    ```
+    An output of 1 is healthy; an output of 0 is unhealthy.
+
+1. Sign into the [Azure portal](https://portal.azure.com), and navigate to your cluster.
+
+1. Under **Monitoring**, select **Metrics**, then select **Keep Alive**, as shown in the following image. An output close to 1 means a healthy cluster.
+
+    ![Cluster Keep Alive metric](media/check-cluster-health/portal-metrics.png)
+
+1. Add other metrics like CPU and Memory Caching to gauge resource utilization for the cluster.
+
+1. If you need assistance diagnosing issues with the health of a cluster, please open a support request in the [Azure portal](https://portal.azure.com).

articles/data-explorer/create-cluster-database-portal.md

@@ -0,0 +1,130 @@
+---
+title: 'Quickstart: Create an Azure Data Explorer cluster and database'
+description: 'In this quickstart, you learn how to create an Azure Data Explorer cluster and database, and ingest (load) data.'
+services: data-explorer
+author: mgblythe
+ms.author: mblythe
+ms.reviewer: mblythe
+ms.service: data-explorer
+ms.topic: quickstart
+ms.date: 09/24/2018
+
+#Customer intent: As a database administrator, I want to create an Azure Data Explorer cluster and database so that I can understand whether Azure Data Explorer is suitable for my analytics projects.
+---
+
+# Quickstart: Create an Azure Data Explorer cluster and database
+
+Azure Data Explorer is a fast and highly scalable data exploration service for log and telemetry data. To use Azure Data Explorer, you first create a *cluster*, and create one or more *databases* in that cluster. Then you *ingest* (load) data into a database so that you can run queries against it. In this quickstart, you create a cluster and a database. In subsequent articles, we show you how to ingest data.
+
+If you don't have an Azure subscription, create a [free Azure account](https://azure.microsoft.com/free/) before you begin.
+
+## Sign in to the Azure portal
+
+Sign in to the [Azure portal](https://portal.azure.com/).
+
+## Create a cluster
+
+You create an Azure Data Explorer cluster in an Azure resource group, with a defined set of compute and storage resources.
+
+1. Select the **Create a resource** button (+) in the upper-left corner of the  portal.
+
+1. Search for *Azure Data Explorer*, then select **Azure Data Explorer**.
+
+   ![Search resources](media/create-cluster-database-portal/search-resources.png)
+
+1. In the lower-right corner, select **Create**.
+
+1. Enter a unique name for your cluster, select your subscription, and create a resource group named *test-resource-group*.
+
+    ![Create resource group](media/create-cluster-database-portal/create-resource-group.png)
+
+1. Fill out the form with the following information.
+
+   ![Create cluster form](media/create-cluster-database-portal/create-cluster-form.png)
+
+    **Setting** | **Suggested value** | **Field description**
+    |---|---|---|
+    | Cluster name | A unique cluster name | Choose a unique name that identifies your cluster. For example, *mytestcluster*. The domain name *[region].kusto.windows.net* is appended to the cluster name you provide. The name can contain only lowercase letters and numbers. It must contain from 3 to 22 characters.
+    | Subscription | Your subscription | Select the Azure subscription that you want to use for your cluster.|
+    | Resource group | *test-resource-group* | Create a new resource group. |
+    | Location | *West US* | Select *West US* for this quickstart. For a production system, select the region that best meets your needs.
+    | Compute specification | *D13_v2* | Select the lowest price specification for this quickstart. For a production system, select the specification that best meets your needs.
+    | | |
+
+1. Select **Create** to provision the cluster. Provisioning typically takes about ten minutes. Select **Notifications** on the toolbar (the bell icon) to monitor the provisioning process.
+
+1. When the process is complete, select **Notifications**, then **Go to resource**.
+
+    ![Go to resource](media/create-cluster-database-portal/notification-resource.png)
+
+## Create a database
+
+You're now ready for the second step in the process: database creation.
+
+1. On the **Overview** tab, select **Create database**.
+
+    ![Step two: create a database](media/create-cluster-database-portal/database-creation.png)
+
+1. Fill out the form with the following information.
+
+    ![Create database form](media/create-cluster-database-portal/create-database.png)
+
+    **Setting** | **Suggested value** | **Field description**
+    |---|---|---|
+    | Database name | *TestDatabase* | The database name must be unique within the cluster.
+    | Retention period | *3650* | The time span for which it's guaranteed that the data is kept available to query. The time span is measured from the time that data is ingested.
+    | Cache period | *31* | The time span for which to keep frequently-queried data available in SSD storage or RAM, rather than in longer-term storage.
+    | | | |
+
+1. Select **Save** to create the database. Creation typically takes less than a minute. When the process is complete, you're back on the cluster **Overview** tab.
+
+## Run basic commands in the database
+
+Now that you have a cluster and database, you can run queries and commands. You don't have any data in the database yet, but you can still see how the tools work.
+
+1. Under your cluster, select **Query**.
+
+    ![Database query](media/create-cluster-database-portal/query-database.png)
+
+1. Paste the following command into the query window: `.show databases`, then select **Run**.
+
+    ![Show databases command](media/create-cluster-database-portal/show-databases.png)
+
+    The result set shows **TestDatabase**, the only database in the cluster.
+
+1. Paste the following command into the query window: `.show tables`, then select that command in the window. Select **Run**.
+
+    This command returns an empty result set because you don't have any tables yet. You add a table in the next article in this series.
+
+## Stop and restart the cluster
+
+You can stop and restart a cluster depending on business needs.
+
+1. To stop the cluster, at the top of the **Overview** tab, select **Stop**.
+
+    When the cluster is stopped, data is not available for queries, and you can't ingest new data.
+
+1. To restart the cluster, at the top of the **Overview** tab, select **Start**.
+
+    When the cluster is restarted, it takes about ten minutes for it to become available (like when it was originally provisioned). It takes additional time for data to load into the hot cache.  
+
+## Clean up resources
+
+If you plan to follow our other quickstarts and tutorials, keep the resources you created. If not, clean up **test-resource-group**, to avoid incurring costs.
+
+1. In the Azure portal, select **Resource groups** on the far left, and then select the resource group you created.  
+
+    If the left menu is collapsed, select ![Expand button](media/create-cluster-database-portal/expand.png) to expand it.
+
+   ![Select resource group to delete](media/create-cluster-database-portal/delete-resources-select.png)
+
+1. Under **test-resource-group**, select **Delete resource group**.
+
+1. In the new window, type the name of the resource group to delete (*test-resource-group*), and then select **Delete**.
+
+## Next steps
+
+> [!div class="nextstepaction"]
+> [Quickstart: Ingest data from Event Hub into Azure Data Explorer](ingest-data-event-hub.md)
+
+

articles/data-explorer/data-explorer-overview.md

@@ -0,0 +1,73 @@
+---
+title: What is Azure Data Explorer?
+description: Azure Data Explorer is a fast and highly scalable data exploration service for log and telemetry data.
+services: data-explorer
+author: orspod
+ms.author: v-orspod
+ms.reviewer: mblythe
+ms.service: data-explorer
+ms.topic: overview
+ms.date: 09/24/2018
+
+#Customer intent: As a data analyst, I want to understand Azure Data Explorer, so I can decide if it's suitable for my analytics workloads.
+---
+
+# What is Azure Data Explorer?
+
+Azure Data Explorer is a fast and highly scalable data exploration service for log and telemetry data. It helps you handle the many data streams emitted by modern software, so you can collect, store, and analyze data. Azure Data Explorer is ideal for analyzing large volumes of diverse data from any data source, such as websites, applications, IoT devices, and more. This data is used for diagnostics, monitoring, reporting, machine learning, and additional analytics capabilities. Azure Data Explorer makes it simple to ingest this data and enables you to perform complex ad-hoc queries on the data in seconds.
+
+## What makes Azure Data Explorer unique?
+
+- Scales quickly to terabytes of data, in minutes, allowing rapid iterations of data exploration to discover relevant insights.
+
+- Offers an innovative query language, optimized for high performance data analytics.
+
+- Supports analysis of high volumes of heterogeneous data (structured and unstructured).
+
+- Provides the ability to build and deploy exactly what you need by combining with other services to supply an encompassing, powerful, and interactive data analytics solution.
+
+## Data warehousing workflow
+
+Azure Data Explorer integrates with other major services to provide an end-to-end solution that includes data collection, ingestion, storage, indexing, querying, and visualization. It has a pivotal role in the data warehousing flow by executing the **EXPLORE** step of the flow on terabytes of diverse raw data.
+
+![Data warehouse diagram](media/data-explorer-overview/data-warehouse.png)
+
+Azure Data Explorer supports several ingestion methods, including connectors to common services like Event Hub, programmatic ingestion using SDKs, such as .NET and Python, and direct access to the engine for exploration purposes. Azure Data Explorer integrates with analytics and modeling services for additional analysis and visualization of data.
+
+## Azure Data Explorer flow
+
+The following diagram shows the different aspects of working with Azure Data Explorer.
+
+![Azure Data Explorer flow](media/data-explorer-overview/workflow.png)
+
+Work in Azure Data Explorer generally follows this pattern:
+
+1. **Create database:** Create a *cluster* and then create one or more *databases* in that cluster. [Quickstart: Create an Azure Data Explorer cluster and database](create-cluster-database-portal.md)
+
+1. **Ingest data:** Load data into database tables so that you can run queries against it. [Quickstart: Ingest data from Event Hub into Azure Data Explorer](ingest-data-event-hub.md)
+
+1. **Query database:** Use our web application to run, review, and share queries and results. It is available in the Azure portal and as a stand-alone application. In addition, you can send queries programmatically (using an SDK) or to a REST API endpoint. [Quickstart: Query data in Azure Data Explorer](web-query-data.md)
+
+## Query experience
+
+A query in Azure Data Explorer is a read-only request to process data and return the results of this processing, without modifying the data or metadata. You continue refining your queries until you have completed your analysis. Azure Data Explorer makes this process easy because of its very fast ad-hoc query experience.
+
+Azure Data Explorer handles large amounts of structured, semi-structured (JSON-like nested types) and unstructured (free-text) data equally well. It allows you to search for specific text terms, locate particular events, and perform metric-style calculations on structured data. Azure Data Explorer bridges the worlds of unstructured text logs and structured numbers and dimensions by extracting values in runtime from free-form text fields. Data exploration is simplified by combining fast text indexing, column store, and time series operations.
+
+Azure Data Explorer capabilities are extended by other services built on its powerful query language, including [Log Analytics](/azure/log-analytics/), [Application Insights](/azure/application-insights/), [Time Series Insights](/azure/time-series-insights/), and [Windows Defender Advanced Threat Protection](/windows/security/threat-protection/windows-defender-atp/windows-defender-advanced-threat-protection/).
+
+## Feedback
+
+We would be thrilled to hear your feedback regarding Azure Data Explorer and its query language at:
+
+- [Ask questions on the forum MSDN](https://social.msdn.microsoft.com/Forums/en-US/home?forum=AzureKusto)
+
+- [Make product suggestions in User Voice](http://aka.ms/AzureDataExplorer.UserVoice)
+
+## Next steps
+
+[Quickstart: Create an Azure Data Explorer cluster and database](create-cluster-database-portal.md)
+
+[Quickstart: Ingest data from Event Hub into Azure Data Explorer](ingest-data-event-hub.md)
+
+[Quickstart: Query data in Azure Data Explorer](web-query-data.md)

articles/data-explorer/index.yml

@@ -0,0 +1,71 @@
+### YamlMime:YamlDocument
+documentType: LandingData
+title: Azure Data Explorer Documentation
+metadata:
+  title: Azure Data Explorer Documentation - Quickstarts, tutorials, and language and API references | Microsoft Docs
+  description: Azure Data Explorer is a log analytics cloud platform optimized for ad-hoc big data queries.
+  author: mgblythe
+  manager: kfile
+  ms.service: data-explorer
+  ms.tgt_pltfrm: na
+  ms.devlang: na
+  ms.topic: landing-page
+  ms.date: 09/24/2018
+  ms.author: mblythe
+abstract:
+  description: <p>Azure Data Explorer is a fast and highly scalable data exploration service for log and telemetry data. It helps you handle the many data streams emitted by modern software, so you can collect, store, and analyze data. Azure Data Explorer is ideal for analyzing large volumes of diverse data from any data source, such as websites, applications, IoT devices, and more.</p><p>Learn how to use Azure Data Explorer with our quickstarts, tutorials, and reference content.</p>
+sections:
+- title: 5-Minute Quickstarts
+  items:
+  - type: paragraph
+    text: 'Learn how to provision and use Azure Data Explorer:'
+  - type: list
+    style: icon48
+    items: 
+    - image: 
+        src: /azure/data-explorer/media/index/portal.svg
+      text: Create a cluster and database
+      href: /azure/data-explorer/create-cluster-database-portal
+    - image: 
+        src: /azure/data-explorer/media/index/portal.svg
+      text: Ingest streaming data from Event Hubs
+      href: /azure/data-explorer/ingest-data-event-hub
+    - image: 
+        src: /azure/data-explorer/media/index/portal.svg
+      text: Query data with the web UI
+      href: /azure/data-explorer/web-query-data
+    - image: 
+        src: /azure/data-explorer/media/index/logo-python.svg
+      text: Ingest queued data with Python
+      href: /azure/data-explorer/python-ingest-data
+    - image: 
+        src: /azure/data-explorer/media/index/logo-python.svg
+      text: Query data with Python
+      href: /azure/data-explorer/python-query-data
+- title: Step-by-Step Tutorials
+  items:
+  - type: paragraph
+    text: 'Learn how to build end-to-end solutions:'
+  - type: list
+    style: unordered
+    items:
+    - html: <a href="/azure/data-explorer/visualize-power-bi">Visualize data from Azure Data Explorer in Power BI</a>
+- title: Free Pluralsight Video Training
+  items:
+  - type: list
+    style: cards
+    className: cardsFTitle
+    items:
+    - title: Query language for Azure Data Explorer
+      href: https://app.pluralsight.com/library/courses/kusto-query-language-kql-from-scratch
+      image: 
+        src: /azure/data-explorer/media/index/logo-pluralsight.svg
+        href: https://app.pluralsight.com/library/courses/kusto-query-language-kql-from-scratch
+- title: Reference
+  items:
+  - type: list
+    style: cards
+    className: cardsD
+    items:
+    - title: Languages and API
+      html: <p><a href="https://aka.ms/kustolangref">Query language, management commands, and APIs</a>