Skip to content

Commit a1f1991

Browse files
committed
Restructuring format and adding screenshot
1 parent 1d8a082 commit a1f1991

File tree

2 files changed

+30
-30
lines changed

2 files changed

+30
-30
lines changed

articles/synapse-analytics/spark/apache-spark-azure-log-analytics.md

Lines changed: 30 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ author: jejiang
55
ms.author: jejiang
66
ms.reviewer: whhender
77
ms.service: azure-synapse-analytics
8-
ms.topic: tutorial
8+
ms.topic: how-to
99
ms.subservice: spark
1010
ms.date: 12/02/2024
1111
ms.custom: references_regions
@@ -22,17 +22,18 @@ Follow these steps to configure the necessary information in Synapse Studio.
2222
### Step 1: Create a Log Analytics workspace
2323

2424
Consult one of the following resources to create this workspace:
25+
2526
- [Create a workspace in the Azure portal.](/azure/azure-monitor/logs/quick-create-workspace)
2627
- [Create a workspace with Azure CLI.](/azure/azure-monitor/logs/resource-manager-workspace)
2728
- [Create and configure a workspace in Azure Monitor by using PowerShell.](/azure/azure-monitor/logs/powershell-workspace-configuration)
2829

29-
### Step 2: Prepare an Apache Spark configuration file
30+
### Step 2: Gather configuration information
3031

31-
Use any of the following options to prepare the file.
32+
Use any of the following options to prepare the configuration.
3233

33-
#### Option 1: Configure with Log Analytics workspace ID and key
34+
#### Option 1: Configure with Log Analytics workspace ID and key
3435

35-
Copy the following Apache Spark configuration, save it as *spark_loganalytics_conf.txt*, and fill in the following parameters:
36+
Gather the following values for the spark configuration:
3637

3738
- `<LOG_ANALYTICS_WORKSPACE_ID>`: Log Analytics workspace ID.
3839
- `<LOG_ANALYTICS_WORKSPACE_KEY>`: Log Analytics key. To find this, in the Azure portal, go to **Azure Log Analytics workspace** > **Agents** > **Primary key**.
@@ -51,13 +52,13 @@ spark.synapse.logAnalytics.secret <LOG_ANALYTICS_WORKSPACE_KEY>
5152
To configure Azure Key Vault to store the workspace key, follow these steps:
5253

5354
1. Create and go to your key vault in the Azure portal.
54-
2. On the settings page for the key vault, select **Secrets**.
55-
3. Select **Generate/Import**.
56-
4. On the **Create a secret** screen, choose the following values:
55+
1. On the settings page for the key vault, select **Secrets**.
56+
1. Select **Generate/Import**.
57+
1. On the **Create a secret** screen, choose the following values:
5758
- **Name**: Enter a name for the secret. For the default, enter `SparkLogAnalyticsSecret`.
5859
- **Value**: Enter the `<LOG_ANALYTICS_WORKSPACE_KEY>` for the secret.
5960
- Leave the other values to their defaults. Then select **Create**.
60-
5. Copy the following Apache Spark configuration, save it as *spark_loganalytics_conf.txt*, and fill in the following parameters:
61+
1. Gather the following values for the spark configuration:
6162

6263
- `<LOG_ANALYTICS_WORKSPACE_ID>`: The Log Analytics workspace ID.
6364
- `<AZURE_KEY_VAULT_NAME>`: The key vault name that you configured.
@@ -81,7 +82,7 @@ spark.synapse.logAnalytics.keyVault.key.secret <AZURE_KEY_VAULT_SECRET_KEY_NAME>
8182
To configure a Key Vault linked service in Synapse Studio to store the workspace key, follow these steps:
8283

8384
1. Follow all the steps in the preceding section, "Option 2."
84-
2. Create a Key Vault linked service in Synapse Studio:
85+
1. Create a Key Vault linked service in Synapse Studio:
8586

8687
a. Go to **Synapse Studio** > **Manage** > **Linked services**, and then select **New**.
8788

@@ -91,7 +92,7 @@ To configure a Key Vault linked service in Synapse Studio to store the workspace
9192

9293
d. Choose your key vault, and select **Create**.
9394

94-
3. Add a `spark.synapse.logAnalytics.keyVault.linkedServiceName` item to the Apache Spark configuration.
95+
1. Add a `spark.synapse.logAnalytics.keyVault.linkedServiceName` item to the Apache Spark configuration.
9596

9697
```properties
9798
spark.synapse.logAnalytics.enabled true
@@ -104,18 +105,21 @@ For a list of Apache Spark configurations, see [Available Apache Spark configura
104105

105106
### Step 3: Create an Apache Spark Configuration
106107

107-
You can create an Apache Spark Configuration to your workspace, and when you create Notebook or Apache spark job definition can select the Apache Spark configuration that you want to use with your Apache Spark pool. When you select it, the details of the configuration are displayed.
108+
You can create an Apache Spark Configuration to your workspace, and when you create Notebook or Apache spark job definition you can select the Apache Spark configuration that you want to use with your Apache Spark pool. When you select it, the details of the configuration are displayed.
108109

109110
1. Select **Manage** > **Apache Spark configurations**.
110-
2. Click on **New** button to create a new Apache Spark configuration, or click on **Import** a local .json file to your workspace.
111-
3. **New Apache Spark configuration** page will be opened after you click on **New** button.
112-
4. For **Name**, you can enter your preferred and valid name.
113-
5. For **Description**, you can input some description in it.
114-
6. For **Annotations**, you can add annotations by clicking the **New** button, and also you can delete existing annotations by selecting and clicking **Delete** button.
115-
7. For **Configuration properties**, customize the configuration by clicking **Add** button to add properties. If you do not add a property, Azure Synapse will use the default value when applicable.
111+
1. Select **New** button to create a new Apache Spark configuration.
112+
1. **New Apache Spark configuration** page will be opened after you select **New** button.
116113

117114
![Screenshot that create spark configuration.](./media/apache-spark-azure-log-analytics/create-spark-configuration.png)
118115

116+
1. For **Name**, you can enter your preferred and valid name.
117+
1. For **Description**, you can input some description in it.
118+
1. For **Annotations**, you can add annotations by clicking the **New** button, and also you can delete existing annotations by selecting and clicking **Delete** button.
119+
1. For **Configuration properties**, add all the properties from the configuration option you chose by selecting the **Add** button. For **Property** add the property name as listed, and for **Value** use the value you gathered during step 2. If you don't add a property, Azure Synapse will use the default value when applicable.
120+
121+
![Screenshot that create spark configuration.](./media/apache-spark-azure-log-analytics/spark-configuration.png)
122+
119123
## Submit an Apache Spark application and view the logs and metrics
120124

121125
Here's how:
@@ -214,8 +218,6 @@ SparkMetrics_CL
214218
| order by TimeGenerated asc
215219
```
216220

217-
218-
219221
## Create and manage alerts
220222

221223
Users can query to evaluate metrics and logs at a set frequency, and fire an alert based on the results. For more information, see [Create, view, and manage log alerts by using Azure Monitor](/azure/azure-monitor/alerts/alerts-log).
@@ -228,25 +230,23 @@ When you want to enable this feature, you need to create managed private endpoin
228230

229231
You can follow below steps to create a managed private endpoint connection to Azure Monitor private link scopes (A M P L S):
230232

231-
1. If there is no existing A M P L S, you can follow [Azure Monitor Private Link connection setup](/azure/azure-monitor/logs/private-link-security) to create one.
232-
2. Navigate to your A M P L S in Azure portal, on the **Azure Monitor Resources** page, click **Add** to add connection to your Azure Log Analytics workspace.
233-
3. Navigate to **Synapse Studio > Manage > Managed private endpoints**, click **New** button, select **Azure Monitor Private Link Scopes**, and **continue**.
233+
1. If there's no existing A M P L S, you can follow [Azure Monitor Private Link connection setup](/azure/azure-monitor/logs/private-link-security) to create one.
234+
1. Navigate to your A M P L S in Azure portal, on the **Azure Monitor Resources** page, select **Add** to add connection to your Azure Log Analytics workspace.
235+
1. Navigate to **Synapse Studio > Manage > Managed private endpoints**, select **New** button, select **Azure Monitor Private Link Scopes**, and **continue**.
234236
> [!div class="mx-imgBorder"]
235237
> ![Screenshot of create A M P L S managed private endpoint 1.](./media/apache-spark-azure-log-analytics/create-ampls-private-endpoint-1.png)
236-
4. Choose your Azure Monitor Private Link Scope you created, and click **Create** button.
238+
1. Choose your Azure Monitor Private Link Scope you created, and select **Create** button.
237239
> [!div class="mx-imgBorder"]
238240
> ![Screenshot of create A M P L S managed private endpoint 2.](./media/apache-spark-azure-log-analytics/create-ampls-private-endpoint-2.png)
239-
5. Wait a few minutes for private endpoint provisioning.
240-
6. Navigate to your A M P L S in Azure portal again, on the **Private Endpoint connections** page, select the connection provisioned and **Approve**.
241+
1. Wait a few minutes for private endpoint provisioning.
242+
1. Navigate to your A M P L S in Azure portal again, on the **Private Endpoint connections** page, select the connection provisioned and **Approve**.
241243

242-
> [!NOTE]
244+
> [!NOTE]
243245
> - The A M P L S object has a number of limits you should consider when planning your Private Link setup. See [A M P L S limits](/azure/azure-monitor/logs/private-link-security) for a deeper review of these limits.
244246
> - Check if you have [right permission](../security/synapse-workspace-access-control-overview.md) to create managed private endpoint.
245247
246-
## Next steps
248+
## Related content
247249

248-
- [Use serverless Apache Spark pool in Synapse Studio](../quickstart-create-apache-spark-pool-studio.md).
249250
- [Run a Spark application in notebook](./apache-spark-development-using-notebooks.md).
250-
- [Create Apache Spark job definition in Azure Studio](./apache-spark-job-definitions.md).
251251
- [Collect Apache Spark applications logs and metrics with Azure Storage account](./azure-synapse-diagnostic-emitters-azure-storage.md).
252252
- [Collect Apache Spark applications logs and metrics with Azure Event Hubs](./azure-synapse-diagnostic-emitters-azure-eventhub.md).
49.3 KB
Loading

0 commit comments

Comments
 (0)