You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/synapse-analytics/spark/azure-synapse-diagnostic-emitters-azure-eventhub.md
+3-6Lines changed: 3 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,12 +16,9 @@ The Synapse Apache Spark diagnostic emitter extension is a library that enables
16
16
17
17
In this tutorial, you learn how to use the Synapse Apache Spark diagnostic emitter extension to emit Apache Spark applications’ logs, event logs, and metrics to your Azure Event Hubs.
18
18
19
-
> [!NOTE]
20
-
> This feature is currently unavailable in the [Azure Synapse Runtime for Apache Spark 3.4](./apache-spark-34-runtime.md) runtime but will be supported post-GA.
21
-
22
19
## Collect logs and metrics to Azure Event Hubs
23
20
24
-
### Step 1: Create an Azure Event Hub instance
21
+
### Step 1: Create an Azure Event Hubs instance
25
22
26
23
To collect diagnostic logs and metrics to Azure Event Hubs, you can use existing Azure Event Hubs instance.
27
24
Or if you don't have one, you can [create an event hub](../../event-hubs/event-hubs-create.md).
Fill in the following parameters in the configuration file: `<connection-string>`.
41
-
For more description of the parameters, you can refer to [Azure EventHub configurations](#available-configurations)
38
+
For more description of the parameters, you can refer to [Azure Event Hubs configurations](#available-configurations).
42
39
43
40
### Step 3: Upload the Apache Spark configuration file to Apache Spark pool
44
41
@@ -53,7 +50,7 @@ For more description of the parameters, you can refer to [Azure EventHub configu
53
50
|`spark.synapse.diagnostic.emitters`| Required. The comma-separated destination names of diagnostic emitters. |
54
51
|`spark.synapse.diagnostic.emitter.<destination>.type`| Required. Built-in destination type. To enable Azure Event Hubs destination, the value should be `AzureEventHub`. |
55
52
|`spark.synapse.diagnostic.emitter.<destination>.categories`| Optional. The comma-separated selected log categories. Available values include `DriverLog`, `ExecutorLog`, `EventLog`, `Metrics`. If not set, the default value is **all** categories. |
56
-
|`spark.synapse.diagnostic.emitter.<destination>.secret`| Optional. The Azure Eventhub instance connection string. This field should match this pattern `Endpoint=sb://<FQDN>/;SharedAccessKeyName=<KeyName>;SharedAccessKey=<KeyValue>;EntityPath=<PathName>`|
53
+
|`spark.synapse.diagnostic.emitter.<destination>.secret`| Optional. The Azure Event Hubs instance connection string. This field should match this pattern `Endpoint=sb://<FQDN>/;SharedAccessKeyName=<KeyName>;SharedAccessKey=<KeyValue>;EntityPath=<PathName>`|
57
54
|`spark.synapse.diagnostic.emitter.<destination>.secret.keyVault`| Required if `.secret` is not specified. The [Azure Key vault](../../key-vault/general/overview.md) name where the secret (connection string) is stored. |
58
55
|`spark.synapse.diagnostic.emitter.<destination>.secret.keyVault.secretName`| Required if `.secret.keyVault` is specified. The Azure Key vault secret name where the secret (connection string) is stored. |
59
56
|`spark.synapse.diagnostic.emitter.<destination>.secret.keyVault.linkedService`| Optional. The Azure Key vault linked service name. When enabled in Synapse pipeline, this is necessary to obtain the secret from AKV. (Please make sure MSI has read permission on the AKV). |
Copy file name to clipboardExpand all lines: articles/synapse-analytics/spark/azure-synapse-diagnostic-emitters-azure-storage.md
-3Lines changed: 0 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -16,9 +16,6 @@ The Synapse Apache Spark diagnostic emitter extension is a library that enables
16
16
17
17
In this tutorial, you learn how to use the Synapse Apache Spark diagnostic emitter extension to emit Apache Spark applications’ logs, event logs, and metrics to your Azure storage account.
18
18
19
-
> [!NOTE]
20
-
> This feature is currently unavailable in the [Azure Synapse Runtime for Apache Spark 3.4](./apache-spark-34-runtime.md) runtime but will be supported post-GA.
0 commit comments