You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
> You need to grant read secret permission to the users who will submit Apache Spark applications. For more information, see [Provide access to Key Vault keys, certificates, and secrets with an Azure role-based access control](/azure/key-vault/general/rbac-guide). When you enable this feature in a Synapse pipeline, you need to use **Option 3**. This is necessary to obtain the secret from Azure Key Vault with workspace managed identity.
60
+
> You need to grant read secret permission to the users who submit Apache Spark applications. For more information, see [Provide access to Key Vault keys, certificates, and secrets with an Azure role-based access control](/azure/key-vault/general/rbac-guide). When you enable this feature in a Synapse pipeline, you need to use **Option 3**. This is necessary to obtain the secret from Azure Key Vault with workspace managed identity.
61
61
62
62
To configure Azure Key Vault to store the workspace key, follow these steps:
63
63
64
64
1. Create and go to your key vault in the Azure portal.
65
-
1. Grant the right permission to the users or workspace managed identites.
65
+
1. Grant the right permission to the users or workspace managed identities.
66
66
1. On the settings page for the key vault, select **Secrets**.
67
67
1. Select **Generate/Import**.
68
68
1. On the **Create a secret** screen, choose the following values:
@@ -146,7 +146,7 @@ You can create an Apache Spark Configuration to your workspace, and when you cre
146
146
1. Select **New** button to create a new Apache Spark configuration.
147
147
1.**New Apache Spark configuration** page will be opened after you select **New** button.
148
148
149
-
:::image type="content" source="./media/apache-spark-azure-log-analytics/create-spark-configuration.png" alt-text="Screenshot that create spark configuration.":::
149
+
:::image type="content" source="./media/apache-spark-azure-log-analytics/create-spark-configuration.png" alt-text="Screenshot that creates Spark configuration.":::
150
150
151
151
1. For **Name**, you can enter your preferred and valid name.
152
152
1. For **Description**, you can input some description in it.
@@ -277,7 +277,7 @@ You can follow below steps to create a managed private endpoint connection to Az
277
277
1. Navigate to your AMPLS in Azure portal again, on the **Private Endpoint connections** page, select the connection provisioned and **Approve**.
278
278
279
279
> [!NOTE]
280
-
> - The AMPLS object has a number of limits you should consider when planning your Private Link setup. See [AMPLS limits](/azure/azure-monitor/logs/private-link-security) for a deeper review of these limits.
280
+
> - The AMPLS object has many limits you should consider when planning your Private Link setup. See [AMPLS limits](/azure/azure-monitor/logs/private-link-security) for a deeper review of these limits.
281
281
> - Check if you have [right permission](../security/synapse-workspace-access-control-overview.md) to create managed private endpoint.
282
282
283
283
## Available configurations
@@ -291,7 +291,7 @@ You can follow below steps to create a managed private endpoint connection to Az
291
291
|`spark.synapse.diagnostic.emitter.<destination>.secret`| Optional. The secret (Log Aanalytics key) content. To find this, in the Azure portal, go to Azure Log Analytics workspace > Agents > Primary key. |
292
292
|`spark.synapse.diagnostic.emitter.<destination>.secret.keyVault`| Required if `.secret` is not specified. The [Azure Key vault](/azure/key-vault/general/overview) name where the secret (AccessKey or SAS) is stored. |
293
293
|`spark.synapse.diagnostic.emitter.<destination>.secret.keyVault.secretName`| Required if `.secret.keyVault` is specified. The Azure Key vault secret name where the secret is stored. |
294
-
|`spark.synapse.diagnostic.emitter.<destination>.secret.keyVault.linkedService`| Optional. The Azure Key vault linked service name. When enabled in Synapse pipeline, this is necessary to obtain the secret from AKV. (Please make sure MSI has read permission on the AKV). |
294
+
|`spark.synapse.diagnostic.emitter.<destination>.secret.keyVault.linkedService`| Optional. The Azure Key vault linked service name. When enabled in Synapse pipeline, this is necessary to obtain the secret from Azure Key vault. (Make sure the MSI has read access to the Azure Key vault). |
295
295
|`spark.synapse.diagnostic.emitter.<destination>.filter.eventName.match`| Optional. The comma-separated Log4j logger names, you can specify which logs to collect. For example `SparkListenerApplicationStart,SparkListenerApplicationEnd`|
296
296
|`spark.synapse.diagnostic.emitter.<destination>.filter.loggerName.match`| Optional. The comma-separated log4j logger names, you can specify which logs to collect. For example: `org.apache.spark.SparkContext,org.example.Logger`|
297
297
|`spark.synapse.diagnostic.emitter.<destination>.filter.metricName.match`| Optional. The comma-separated spark metric name suffixes, you can specify which metrics to collect. For example:`jvm.heap.used`|
-If you are new to Azure Event Hubs, read through [Event Hubs overview](/azure/event-hubs/event-hubs-about) and [Event Hubs features](/azure/event-hubs/event-hubs-features).
15
+
- An Azure subscription. You can also [create a free account](https://azure.microsoft.com/free/) before you get started.
1. Sign in to the [Azure portal](https://portal.azure.com/) and go to [App registrations](/entra/identity-platform/quickstart-register-app#register-an-application).
29
29
2. Create a new app registration for your Synapse workspace.
30
30
31
-
:::image type="content" source="media\how-to-use-certsp-emit-log-to-eventhub\create-a-new-app-registration.png" alt-text="Screenshot showing create a new app registration.":::
31
+
:::image type="content" source="media\how-to-use-certificate-with-service-principalp-emit-log-event-hubs\create-a-new-app-registration.png" alt-text="Screenshot showing create a new app registration.":::
32
32
33
-
## Step 2. Generate a Certificate in Key Vault
33
+
## Step 2. Generate a certificate in Key Vault
34
34
35
35
1. Navigate to Key Vault.
36
36
2. Expand the **Object**, and select the **Certificates**.
37
37
3. Click on **Generate/Import**.
38
38
39
-
:::image type="content" source="media\how-to-use-certsp-emit-log-to-eventhub\generate-a-new-certificate.png" alt-text="Screenshot showing generate a new certificate for app.":::
39
+
:::image type="content" source="media\how-to-use-certificate-with-service-principalp-emit-log-event-hubs\generate-a-new-certificate.png" alt-text="Screenshot showing generate a new certificate for app.":::
40
40
41
-
## Step 3. Trust the Certificate in the Application
41
+
## Step 3. Trust the certificate in the application
42
42
43
43
1. Go to the app created in Step 1 -> **Manage** -> **Manifest**.
44
44
2. Append the certificate details to the manifest file to establish trust.
@@ -53,27 +53,27 @@ ms.date: 03/24/2025
53
53
]
54
54
```
55
55
56
-
:::image type="content" source="media\how-to-use-certsp-emit-log-to-eventhub\trust-the-certificate.png" alt-text="Screenshot showing trust the certificate in the application.":::
56
+
:::image type="content" source="media\how-to-use-certificate-with-service-principalp-emit-log-event-hubs\trust-the-certificate.png" alt-text="Screenshot showing trust the certificate in the application.":::
57
57
58
58
## Step 4. Assign Azure Event Hubs Data Sender Role
59
59
60
-
1. In Event Hub, navigate to Access control (IAM).
61
-
2. Assign the app (Service Principal) with the Azure Event Hubs Data Sender role.
60
+
1. In Azure Event Hubs, navigate to Access control (IAM).
61
+
2. Assign the Azure Event Hubs data sender role to the application (service principal).
1. In Synapse Analytics workspace, go to **Manage** -> **Linked service**.
68
-
2. Create a new **Linked Service** in Synapse to connect to **Key Vault**.
67
+
1. In Synapse Analytics workspace, go to **Manage** -> **linked service**.
68
+
2. Create a new **linked Service** in Synapse to connect to **Key Vault**.
69
69
70
-
:::image type="content" source="media\how-to-use-certsp-emit-log-to-eventhub\create-a-linked-service-in-synapse.png" alt-text="Screenshot showing create a linked service in synapse.":::
70
+
:::image type="content" source="media\how-to-use-certificate-with-service-principalp-emit-log-event-hubs\create-a-linked-service-in-synapse.png" alt-text="Screenshot showing create a linked service in synapse.":::
71
71
72
-
## Step 6. Assign Reader Role to Linked Service in Key Vault
72
+
## Step 6. Assign reader role to linked service in Key Vault
73
73
74
-
1. Get the workspace managed identity ID from the linked service. The **managed identity name** and **object ID** for the linked service is under **Edit Linked Service**.
74
+
1. Get the workspace managed identity ID from the linked service. The **managed identity name** and **object ID** for the linked service is under **Edit linked service**.
75
75
76
-
:::image type="content" source="media\how-to-use-certsp-emit-log-to-eventhub\managed-identity-name-and-object-id.png" alt-text="Screenshot showing managed identity name and object id are in edit linked service.":::
76
+
:::image type="content" source="media\how-to-use-certificate-with-service-principalp-emit-log-event-hubs\managed-identity-name-and-object-id.png" alt-text="Screenshot showing managed identity name and object ID are in edit linked service.":::
77
77
78
78
2. In **Key Vault**, assign the linked service a **Reader** role.
79
79
@@ -84,10 +84,10 @@ Gather the following values and add to the Apache Spark configuration.
84
84
- **<EMITTER_NAME>**: The name for the emmiter.
85
85
- **<CERTIFICATE_NAME>**: The certificate name that you generated in the key vault.
86
86
- **<LINKED_SERVICE_NAME>**: The Azure Key vault linked service name.
87
-
- **<EVENT_HUB_HOST_NAME>**: The Event Hub host name, you can find it in Event Hubs Namespace -> Overview -> Host name.
88
-
- **<SERVICE_PRINCIPAL_TENANT_ID>**: The service principal tenant id, you can find it in App registrations -> your app name -> Overview -> Directory (tenant) ID
89
-
- **<SERVICE_PRINCIPAL_CLIENT_ID>**: The service principal client id, you can find it in registrations -> your app name -> Overview -> Application(client) ID
90
-
- **<EVENT_HUB_ENTITY_PATH>**: The Event Hub entity path, you can find it in Event Hubs Namespace -> Overview -> Host name.
87
+
- **<EVENT_HUB_HOST_NAME>**: The Azure Event Hubs host name, you can find it in Azure Event Hubs Namespace -> Overview -> Host name.
88
+
- **<SERVICE_PRINCIPAL_TENANT_ID>**: The service principal tenant ID, you can find it in App registrations -> your app name -> Overview -> Directory (tenant) ID
89
+
- **<SERVICE_PRINCIPAL_CLIENT_ID>**: The service principal client ID, you can find it in registrations -> your app name -> Overview -> Application(client) ID
90
+
- **<EVENT_HUB_ENTITY_PATH>**: The Azure Event Hubs entity path, you can find it in Azure Event Hubs Namespace -> Overview -> Host name.
0 commit comments