Skip to content

Commit bcd0c3f

Browse files
committed
Security warning update
1 parent 7adb5de commit bcd0c3f

File tree

2 files changed

+32
-23
lines changed

2 files changed

+32
-23
lines changed

articles/synapse-analytics/data-explorer/ingest-data/data-explorer-ingest-event-hub-overview.md

Lines changed: 32 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
2-
title: Event Hub data connection for Azure Synapse Data Explorer (Preview)
3-
description: This article provides an overview of how to ingest (load) data into Azure Synapse Data Explorer from Event Hub.
2+
title: Event Hubs data connection for Azure Synapse Data Explorer (Preview)
3+
description: This article provides an overview of how to ingest (load) data into Azure Synapse Data Explorer from Event Hubs.
44
ms.topic: how-to
55
ms.date: 11/02/2021
66
author: shsagir
@@ -9,17 +9,17 @@ ms.reviewer: tzgitlin
99
ms.service: azure-synapse-analytics
1010
ms.subservice: data-explorer
1111
---
12-
# Event Hub data connection (Preview)
12+
# Event Hubs data connection (Preview)
1313

1414
[Azure Event Hubs](../../../event-hubs/event-hubs-about.md) is a big data streaming platform and event ingestion service. Azure Synapse Data Explorer offers continuous ingestion from customer-managed Event Hubs.
1515

16-
The Event Hub ingestion pipeline transfers events to Azure Synapse Data Explorer in several steps. You first create an Event Hub in the Azure portal. You then create a target table in Azure Synapse Data Explorer into which the [data in a particular format](#data-format), will be ingested using the given [ingestion properties](#ingestion-properties). The Event Hub connection needs to know [events routing](#events-routing). Data is embedded with selected properties according to the [event system properties mapping](#event-system-properties-mapping). [Create a connection](#event-hub-connection) to Event Hub to [create an Event Hub](#create-an-event-hub) and [send events](#send-events). This process can be managed through the [Azure portal](data-explorer-ingest-event-hub-portal.md), programmatically with [C#](data-explorer-ingest-event-hub-csharp.md) or [Python](data-explorer-ingest-event-hub-python.md), or with the [Azure Resource Manager template](data-explorer-ingest-event-hub-resource-manager.md).
16+
The Event Hubs ingestion pipeline transfers events to Azure Synapse Data Explorer in several steps. You first create an Event Hubs in the Azure portal. You then create a target table in Azure Synapse Data Explorer into which the [data in a particular format](#data-format), will be ingested using the given [ingestion properties](#ingestion-properties). The Event Hubs connection needs to know [events routing](#events-routing). Data is embedded with selected properties according to the [event system properties mapping](#event-system-properties-mapping). [Create a connection](#event-hub-connection) to Event Hubs to [create an Event Hubs](#create-an-event-hub) and [send events](#send-events). This process can be managed through the [Azure portal](data-explorer-ingest-event-hub-portal.md), programmatically with [C#](data-explorer-ingest-event-hub-csharp.md) or [Python](data-explorer-ingest-event-hub-python.md), or with the [Azure Resource Manager template](data-explorer-ingest-event-hub-resource-manager.md).
1717

1818
For general information about data ingestion in Azure Synapse Data Explorer, see [Azure Synapse Data Explorer data ingestion overview](data-explorer-ingest-data-overview.md).
1919

2020
## Data format
2121

22-
* Data is read from the Event Hub in form of [EventData](/dotnet/api/microsoft.servicebus.messaging.eventdata) objects.
22+
* Data is read from the Event Hubs in form of [EventData](/dotnet/api/microsoft.servicebus.messaging.eventdata) objects.
2323
* See [supported formats](data-explorer-ingest-data-supported-formats.md).
2424
> [!NOTE]
2525
> Event Hub doesn't support the .raw format.
@@ -46,12 +46,22 @@ Ingestion properties instruct the ingestion process, where to route the data, an
4646
4747
## Events routing
4848

49-
When you set up an Event Hub connection to Azure Synapse Data Explorer cluster, you specify target table properties (table name, data format, compression, and mapping). The default routing for your data is also referred to as `static routing`.
49+
When you set up an Event Hubs connection to Azure Synapse Data Explorer cluster, you specify target table properties (table name, data format, compression, and mapping). The default routing for your data is also referred to as `static routing`.
5050
You can also specify target table properties for each event, using event properties. The connection will dynamically route the data as specified in the [EventData.Properties](/dotnet/api/microsoft.servicebus.messaging.eventdata.properties#Microsoft_ServiceBus_Messaging_EventData_Properties), overriding the static properties for this event.
5151

52-
In the following example, set Event Hub details and send weather metric data to table `WeatherMetrics`.
52+
In the following example, set Event Hubs details and send weather metric data to table `WeatherMetrics`.
5353
Data is in `json` format. `mapping1` is pre-defined on the table `WeatherMetrics`.
5454

55+
>[!WARNING]
56+
>This example uses connection string authentication to connect to Event Hubs for simplicity of the example. However, hard-coding a connection string into your script requires a very high degree of trust in the application, and carries security risks.
57+
>
58+
>For long-term, secure solutions, use one of these options:
59+
>
60+
>* [Passwordless authentication](../../../event-hubs/event-hubs-dotnet-standard-getstarted-send.md?tabs=passwordless)
61+
>* [Store your connection string in an Azure Key Vault](/key-vault/secrets/quick-create-net.md#retrieve-a-secret)
62+
63+
Microsoft recommends that you use the most secure authentication flow available. The authentication flow described in this procedure requires a very high degree of trust in the application, and carries risks that aren't present in other flows. You should only use this flow when other more secure flows, such as managed identities, aren't viable.
64+
5565
```csharp
5666
var eventHubNamespaceConnectionString=<connection_string>;
5767
var eventHubName=<event_hub>;
@@ -75,19 +85,19 @@ eventHubClient.Close();
7585

7686
## Event system properties mapping
7787

78-
System properties store properties that are set by the Event Hubs service, at the time the event is enqueued. The Azure Synapse Data Explorer Event Hub connection will embed the selected properties into the data landing in your table.
88+
System properties store properties that are set by the Event Hubs service, at the time, the event is enqueued. The Azure Synapse Data Explorer Event Hubs connection will embed the selected properties into the data landing in your table.
7989

8090
[!INCLUDE [event-hub-system-mapping](../includes/data-explorer-event-hub-system-mapping.md)]
8191

8292
### System properties
8393

84-
Event Hub exposes the following system properties:
94+
Event Hubs exposes the following system properties:
8595

8696
|Property |Data Type |Description|
8797
|---|---|---|
8898
| x-opt-enqueued-time |datetime | UTC time when the event was enqueued |
89-
| x-opt-sequence-number |long | The logical sequence number of the event within the partition stream of the Event Hub
90-
| x-opt-offset |string | The offset of the event from the Event Hub partition stream. The offset identifier is unique within a partition of the Event Hub stream |
99+
| x-opt-sequence-number |long | The logical sequence number of the event within the partition stream of the Event Hubs
100+
| x-opt-offset |string | The offset of the event from the Event Hubs partition stream. The offset identifier is unique within a partition of the Event Hubs stream |
91101
| x-opt-publisher |string | The publisher name, if the message was sent to a publisher endpoint |
92102
| x-opt-partition-key |string |The partition key of the corresponding partition that stored the event |
93103

@@ -97,36 +107,36 @@ If you selected **Event system properties** in the **Data Source** section of th
97107

98108
[!INCLUDE [data-explorer-container-system-properties](../includes/data-explorer-container-system-properties.md)]
99109

100-
## Event Hub connection
110+
## Event Hubs connection
101111

102112
> [!Note]
103113
> For best performance, create all resources in the same region as the Azure Synapse Data Explorer cluster.
104114
105-
### Create an Event Hub
115+
### Create an Event Hubs
106116

107-
If you don't already have one, [Create an Event Hub](../../../event-hubs/event-hubs-create.md). Connecting to Event Hub can be managed through the [Azure portal](data-explorer-ingest-event-hub-portal.md), programmatically with [C#](data-explorer-ingest-event-hub-csharp.md) or [Python](data-explorer-ingest-event-hub-python.md), or with the [Azure Resource Manager template](data-explorer-ingest-event-hub-resource-manager.md).
117+
If you don't already have one, [Create an Event Hubs](../../../event-hubs/event-hubs-create.md). Connecting to Event Hubs can be managed through the [Azure portal](data-explorer-ingest-event-hub-portal.md), programmatically with [C#](data-explorer-ingest-event-hub-csharp.md) or [Python](data-explorer-ingest-event-hub-python.md), or with the [Azure Resource Manager template](data-explorer-ingest-event-hub-resource-manager.md).
108118

109119
> [!Note]
110120
> * The partition count isn't changeable, so you should consider long-term scale when setting partition count.
111121
> * Consumer group *must* be unique per consumer. Create a consumer group dedicated to Azure Synapse Data Explorer connection.
112122
113123
## Send events
114124

115-
See the [sample app](https://github.com/Azure-Samples/event-hubs-dotnet-ingest) that generates data and sends it to an Event Hub.
125+
See the [sample app](https://github.com/Azure-Samples/event-hubs-dotnet-ingest) that generates data and sends it to an Event Hubs.
116126

117-
For an example of how to generate sample data, see [Ingest data from Event Hub into Azure Synapse Data Explorer](data-explorer-ingest-event-hub-portal.md#generate-sample-data)
127+
For an example of how to generate sample data, see [Ingest data from Event Hubs into Azure Synapse Data Explorer](data-explorer-ingest-event-hub-portal.md#generate-sample-data)
118128

119129
## Set up Geo-disaster recovery solution
120130

121-
Event Hub offers a [Geo-disaster recovery](../../../event-hubs/event-hubs-geo-dr.md) solution.
122-
Azure Synapse Data Explorer doesn't support `Alias` Event Hub namespaces. To implement the Geo-disaster recovery in your solution, create two Event Hub data connections: one for the primary namespace and one for the secondary namespace. Azure Synapse Data Explorer will listen to both Event Hub connections.
131+
Event Hubs offers a [Geo-disaster recovery](../../../event-hubs/event-hubs-geo-dr.md) solution.
132+
Azure Synapse Data Explorer doesn't support `Alias` Event Hubs namespaces. To implement the Geo-disaster recovery in your solution, create two Event Hubs data connections: one for the primary namespace and one for the secondary namespace. Azure Synapse Data Explorer will listen to both Event Hubs connections.
123133

124134
> [!NOTE]
125135
> It's the user's responsibility to implement a failover from the primary namespace to the secondary namespace.
126136
127137
## Next steps
128138

129-
- [Ingest data from Event Hub into Azure Synapse Data Explorer](data-explorer-ingest-event-hub-portal.md)
130-
- [Create an Event Hub data connection for Azure Synapse Data Explorer using C#](data-explorer-ingest-event-hub-csharp.md)
131-
- [Create an Event Hub data connection for Azure Synapse Data Explorer using Python](data-explorer-ingest-event-hub-python.md)
132-
- [Create an Event Hub data connection for Azure Synapse Data Explorer using Azure Resource Manager template](data-explorer-ingest-event-hub-resource-manager.md)
139+
- [Ingest data from Event Hubs into Azure Synapse Data Explorer](data-explorer-ingest-event-hub-portal.md)
140+
- [Create an Event Hubs data connection for Azure Synapse Data Explorer using C#](data-explorer-ingest-event-hub-csharp.md)
141+
- [Create an Event Hubs data connection for Azure Synapse Data Explorer using Python](data-explorer-ingest-event-hub-python.md)
142+
- [Create an Event Hubs data connection for Azure Synapse Data Explorer using Azure Resource Manager template](data-explorer-ingest-event-hub-resource-manager.md)

articles/synapse-analytics/how-to-move-workspace-from-one-region-to-another.md

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -93,7 +93,6 @@ New-AzStorageAccount -ResourceGroupName $resourceGroupName `
9393
-EnableHierarchicalNamespace $true
9494
```
9595

96-
9796
#### Create an Azure Synapse workspace
9897

9998
```powershell

0 commit comments

Comments
 (0)