You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/hdinsight/apache-kafka-spark-structured-streaming-cosmosdb.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -26,7 +26,7 @@ Spark structured streaming is a stream processing engine built on Spark SQL. It
26
26
27
27
Apache Kafka on HDInsight doesn't provide access to the Kafka brokers over the public internet. Anything that talks to Kafka must be in the same Azure virtual network as the nodes in the Kafka cluster. For this example, both the Kafka and Spark clusters are located in an Azure virtual network. The following diagram shows how communication flows between the clusters:
28
28
29
-
:::image type="content" source="./media/apache-kafka-spark-structured-streaming-cosmosdb/apache-spark-kafka-vnet.png" alt-text="Diagram of Spark and Kafka clusters in an Azure virtual network" border="false":::
29
+
:::image type="content" source="./media/apache-kafka-spark-structured-streaming-cosmosdb/apache-spark-kafka-vnet.png" alt-text="Diagram of Spark and Kafka clusters in an Azure virtual network." border="false":::
30
30
31
31
> [!NOTE]
32
32
> The Kafka service is limited to communication within the virtual network. Other services on the cluster, such as SSH and Ambari, can be accessed over the internet. For more information on the public ports available with HDInsight, see [Ports and URIs used by HDInsight](hdinsight-hadoop-port-settings-for-services.md).
@@ -68,7 +68,7 @@ While you can create an Azure virtual network, Kafka, and Spark clusters manuall
68
68
|Ssh User Name|The SSH user to create for the Spark and Kafka clusters.|
69
69
|Ssh Password|The password for the SSH user for the Spark and Kafka clusters.|
70
70
71
-
:::image type="content" source="./media/apache-kafka-spark-structured-streaming-cosmosdb/hdi-custom-parameters-40.png" alt-text="HDInsight version 4.0 custom deployment values":::
71
+
:::image type="content" source="./media/apache-kafka-spark-structured-streaming-cosmosdb/hdi-custom-parameters-40.png" alt-text="HDInsight version 4.0 custom deployment values.":::
72
72
73
73
1. Read the **Terms and Conditions**, and then select **I agree to the terms and conditions stated above**.
Copy file name to clipboardExpand all lines: articles/hdinsight/cluster-availability-monitor-logs.md
+13-13Lines changed: 13 additions & 13 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -20,15 +20,15 @@ As a prerequisite, you'll need a Log Analytics Workspace to store the collected
20
20
21
21
From the HDInsight cluster resource page in the portal, select **Azure Monitor**. Then, select **enable** and select your Log Analytics workspace from the drop-down.
By default, this installs the OMS agent on all of the cluster nodes except for edge nodes. Because no OMS agent is installed on cluster edge nodes, there is no telemetry on edge nodes present in Log Analytics by default.
26
26
27
27
## Query metrics and logs tables
28
28
29
29
Once Azure Monitor log integration is enabled (this may take a few minutes), navigate to your **Log Analytics Workspace** resource and select **Logs**.
@@ -42,7 +42,7 @@ Logs list a number of sample queries, such as:
42
42
43
43
As an example, run the **Availability rate** sample query by selecting **Run** on that query, as shown in the screenshot above. This will show the availability rate of each node in your cluster as a percentage. If you have enabled multiple HDInsight clusters to send metrics to the same Log Analytics workspace, you'll see the availability rate for all nodes (excluding edge nodes) in those clusters displayed.
> Availability rate is measured over a 24-hour period, so your cluster will need to run for at least 24 hours before you see accurate availability rates.
@@ -55,16 +55,16 @@ You can also set up Azure Monitor alerts that will trigger when the value of a m
55
55
56
56
From **Logs**, run the **Unavailable computers** sample query by selecting **Run** on that query, as shown below.
If all nodes are available, this query should return zero results for now. Click **New alert rule** to begin configuring your alert for this query.
61
61
62
-
:::image type="content" source="media/cluster-availability-monitor-logs/portal-logs-new-alert-rule.png" alt-text="Log Analytics workspace new alert rule":::
62
+
:::image type="content" source="media/cluster-availability-monitor-logs/portal-logs-new-alert-rule.png" alt-text="Log Analytics workspace new alert rule.":::
63
63
64
64
There are three components to an alert: the *resource* for which to create the rule (the Log Analytics workspace in this case), the *condition* to trigger the alert, and the *action groups* that determine what will happen when the alert is triggered.
65
65
Click the **condition title**, as shown below, to finish configuring the signal logic.
@@ -80,11 +80,11 @@ For the purpose of this alert, you want to make sure **Period=Frequency.** More
80
80
81
81
Select **Done** when you're finished configuring the signal logic.
82
82
83
-
:::image type="content" source="media/cluster-availability-monitor-logs/portal-configure-signal-logic.png" alt-text="Alert rule configures signal logic":::
83
+
:::image type="content" source="media/cluster-availability-monitor-logs/portal-configure-signal-logic.png" alt-text="Alert rule configures signal logic.":::
84
84
85
85
If you don't already have an existing action group, click **Create New** under the **Action Groups** section.
86
86
87
-
:::image type="content" source="media/cluster-availability-monitor-logs/portal-create-new-action-group.png" alt-text="Alert rule creates new action group":::
87
+
:::image type="content" source="media/cluster-availability-monitor-logs/portal-create-new-action-group.png" alt-text="Alert rule creates new action group.":::
88
88
89
89
This will open **Add action group**. Choose an **Action group name**, **Short name**, **Subscription**, and **Resource group.** Under the **Actions** section, choose an **Action Name** and select **Email/SMS/Push/Voice** as the **Action Type.**
90
90
@@ -93,26 +93,26 @@ This will open **Add action group**. Choose an **Action group name**, **Short na
93
93
94
94
This will open **Email/SMS/Push/Voice**. Choose a **Name** for the recipient, **check** the **Email** box, and type an email address to which you want the alert sent. Select **OK** in **Email/SMS/Push/Voice**, then in **Add action group** to finish configuring your action group.
After these blades close, you should see your action group listed under the **Action Groups** section. Finally, complete the **Alert Details** section by typing an **Alert Rule Name** and **Description** and choosing a **Severity**. Click **Create Alert Rule** to finish.
> The ability to specify **Severity** is a powerful tool that can be used when creating multiple alerts. For example, you could create one alert to raise a Warning (Sev 1) if a single head node goes down and another alert that raises Critical (Sev 0) in the unlikely event that both head nodes go down.
104
104
105
105
When the condition for this alert is met, the alert will fire and you'll receive an email with the alert details like this:
Selecting on a severity grouping (i.e. **Sev 1,** as highlighted above) will show records for all alerts of that severity that have fired like below:
114
114
115
-
:::image type="content" source="media/cluster-availability-monitor-logs/portal-oms-alerts-sev1.png" alt-text="Log Analytics workspace sev one alert":::
115
+
:::image type="content" source="media/cluster-availability-monitor-logs/portal-oms-alerts-sev1.png" alt-text="Log Analytics workspace sev one alert.":::
Copy file name to clipboardExpand all lines: articles/hdinsight/connect-on-premises-network.md
+6-6Lines changed: 6 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -32,7 +32,7 @@ These configurations enable the following behavior:
32
32
33
33
In the following diagram, green lines are requests for resources that end in the DNS suffix of the virtual network. Blue lines are requests for resources in the on-premises network or on the public internet.
34
34
35
-
:::image type="content" source="./media/connect-on-premises-network/on-premises-to-cloud-dns.png" alt-text="Diagram of how DNS requests are resolved in the configuration" border="false":::
35
+
:::image type="content" source="./media/connect-on-premises-network/on-premises-to-cloud-dns.png" alt-text="Diagram of how DNS requests are resolved in the configuration." border="false":::
36
36
37
37
## Prerequisites
38
38
@@ -59,7 +59,7 @@ These steps use the [Azure portal](https://portal.azure.com) to create an Azure
59
59
60
60
1. From the top menu, select **+ Create a resource**.
61
61
62
-
:::image type="content" source="./media/connect-on-premises-network/azure-portal-create-resource.png" alt-text="Create an Ubuntu virtual machine":::
62
+
:::image type="content" source="./media/connect-on-premises-network/azure-portal-create-resource.png" alt-text="Create an Ubuntu virtual machine.":::
63
63
64
64
1. Select **Compute** > **Virtual machine** to go to the **Create a virtual machine** page.
65
65
@@ -78,7 +78,7 @@ These steps use the [Azure portal](https://portal.azure.com) to create an Azure
78
78
|Password or SSH public key | The available field is determined by your choice for **Authentication type**. Enter the appropriate value.|
79
79
|Public inbound ports|Select **Allow selected ports**. Then select **SSH (22)** from the **Select inbound ports** drop-down list.|
Copy file name to clipboardExpand all lines: articles/hdinsight/control-network-traffic.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,7 +16,7 @@ Network traffic in an Azure Virtual Networks can be controlled using the followi
16
16
17
17
As a managed service, HDInsight requires unrestricted access to the HDInsight health and management services both for incoming and outgoing traffic from the VNET. When using NSGs, you must ensure that these services can still communicate with HDInsight cluster.
18
18
19
-
:::image type="content" source="./media/control-network-traffic/hdinsight-vnet-diagram.png" alt-text="Diagram of HDInsight entities created in Azure custom VNET" border="false":::
19
+
:::image type="content" source="./media/control-network-traffic/hdinsight-vnet-diagram.png" alt-text="Diagram of HDInsight entities created in Azure custom VNET." border="false":::
0 commit comments