Skip to content

Commit 07396e8

Browse files
authored
Merge pull request #189902 from HeidiSteen/heidist-work
Azure SQL define data source section
2 parents 2aac8fe + fb1dead commit 07396e8

File tree

1 file changed

+41
-35
lines changed

1 file changed

+41
-35
lines changed

articles/search/search-howto-connecting-azure-sql-database-to-azure-search-using-indexers.md

Lines changed: 41 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,9 @@ Incremental indexing is possible. If you have a large data set and plan to run t
2929

3030
## Define the data source
3131

32-
1. Create the data source:
32+
The data source definition specifies the data to index, credentials, and policies for identifying changes in the data. A data source is defined as an independent resource so that it can be used by multiple indexers.
33+
34+
1. [Create or update a data source](/rest/api/searchservice/create-data-source) to set its definition:
3335

3436
```http
3537
POST https://myservice.search.windows.net/datasources?api-version=2020-06-30
@@ -44,9 +46,15 @@ Incremental indexing is possible. If you have a large data set and plan to run t
4446
}
4547
```
4648

47-
The connection string can follow either of the below formats:
48-
1. You can get the connection string from the [Azure portal](https://portal.azure.com); use the `ADO.NET connection string` option.
49-
1. A managed identity connection string that does not include an account key with the following format: `Initial Catalog|Database=<your database name>;ResourceId=/subscriptions/<your subscription ID>/resourceGroups/<your resource group name>/providers/Microsoft.Sql/servers/<your SQL Server name>/;Connection Timeout=connection timeout length;`. To use this connection string, follow the instructions for [Setting up an indexer connection to an Azure SQL Database using a managed identity](search-howto-managed-identities-sql.md).
49+
1. Set "type" to `"azuresql"` (required).
50+
51+
1. Set "credentials" to a connection string:
52+
53+
+ You can get the connection string from the [Azure portal](https://portal.azure.com). Use the `ADO.NET connection string` option.
54+
55+
+ You can specify a managed identity connection string that does not include database secrets with the following format: `Initial Catalog|Database=<your database name>;ResourceId=/subscriptions/<your subscription ID>/resourceGroups/<your resource group name>/providers/Microsoft.Sql/servers/<your SQL Server name>/;Connection Timeout=connection timeout length;`.
56+
57+
To use this connection string, follow the instructions for [Setting up an indexer connection to an Azure SQL Database using a managed identity](search-howto-managed-identities-sql.md).
5058

5159
## Add search fields to an index
5260

@@ -200,34 +208,33 @@ Within an indexer definition, you can specify a change detection policies that t
200208

201209
### SQL Integrated Change Tracking Policy
202210

203-
We recommend using **SQL Integrated Change Tracking Policy** for its efficiency and its ability to identify deleted rows.
211+
We recommend using "SqlIntegratedChangeTrackingPolicy" for its efficiency and its ability to identify deleted rows.
204212

205-
#### Requirements
213+
Database requirements:
206214

207-
+ Database version requirements:
208-
* SQL Server 2012 SP3 and later, if you're using SQL Server on Azure VMs.
209-
* Azure SQL Database or SQL Managed Instance.
210-
+ Tables only (no views).
211-
+ On the database, [enable change tracking](/sql/relational-databases/track-changes/enable-and-disable-change-tracking-sql-server) for the table.
212-
+ No composite primary key (a primary key containing more than one column) on the table.
215+
+ SQL Server 2012 SP3 and later, if you're using SQL Server on Azure VMs
216+
+ Azure SQL Database or SQL Managed Instance
217+
+ Tables only (no views).
218+
+ On the database, [enable change tracking](/sql/relational-databases/track-changes/enable-and-disable-change-tracking-sql-server) for the table
219+
+ No composite primary key (a primary key containing more than one column) on the table
213220

214-
#### Usage
215-
216-
To use this policy, create or update your data source like this:
221+
Change detection policies are added to data source definitions. To use this policy, create or update your data source like this:
217222

218223
```http
224+
POST https://myservice.search.windows.net/datasources?api-version=2020-06-30
225+
Content-Type: application/json
226+
api-key: admin-key
219227
{
220228
"name" : "myazuresqldatasource",
221229
"type" : "azuresql",
222230
"credentials" : { "connectionString" : "connection string" },
223-
"container" : { "name" : "table or view name" },
231+
"container" : { "name" : "table name" },
224232
"dataChangeDetectionPolicy" : {
225-
"@odata.type" : "#Microsoft.Azure.Search.SqlIntegratedChangeTrackingPolicy"
226-
}
233+
"@odata.type" : "#Microsoft.Azure.Search.SqlIntegratedChangeTrackingPolicy"
227234
}
228235
```
229236

230-
When using SQL integrated change tracking policy, do not specify a separate data deletion detection policy - this policy has built-in support for identifying deleted rows. However, for the deletes to be detected "automagically", the document key in your search index must be the same as the primary key in the SQL table.
237+
When using SQL integrated change tracking policy, do not specify a separate data deletion detection policy. The SQL integrated change tracking policy has built-in support for identifying deleted rows. However, for the deletes to be detected automatically, the document key in your search index must be the same as the primary key in the SQL table.
231238

232239
> [!NOTE]
233240
> When using [TRUNCATE TABLE](/sql/t-sql/statements/truncate-table-transact-sql) to remove a large number of rows from a SQL table, the indexer needs to be [reset](/rest/api/searchservice/reset-indexer) to reset the change tracking state to pick up row deletions.
@@ -236,39 +243,38 @@ When using SQL integrated change tracking policy, do not specify a separate data
236243

237244
### High Water Mark Change Detection policy
238245

239-
This change detection policy relies on a "high water mark" column capturing the version or time when a row was last updated. If you're using a view, you must use a high water mark policy. The high water mark column must meet the following requirements.
246+
This change detection policy relies on a "high water mark" column in your table or view that captures the version or time when a row was last updated. If you're using a view, you must use a high water mark policy.
240247

241-
#### Requirements
248+
The high water mark column must meet the following requirements:
242249

243-
* All inserts specify a value for the column.
244-
* All updates to an item also change the value of the column.
245-
* The value of this column increases with each insert or update.
246-
* Queries with the following WHERE and ORDER BY clauses can be executed efficiently: `WHERE [High Water Mark Column] > [Current High Water Mark Value] ORDER BY [High Water Mark Column]`
250+
+ All inserts specify a value for the column.
251+
+ All updates to an item also change the value of the column.
252+
+ The value of this column increases with each insert or update.
253+
+ Queries with the following WHERE and ORDER BY clauses can be executed efficiently: `WHERE [High Water Mark Column] > [Current High Water Mark Value] ORDER BY [High Water Mark Column]`
247254

248-
> [!IMPORTANT]
255+
> [!NOTE]
249256
> We strongly recommend using the [rowversion](/sql/t-sql/data-types/rowversion-transact-sql) data type for the high water mark column. If any other data type is used, change tracking is not guaranteed to capture all changes in the presence of transactions executing concurrently with an indexer query. When using **rowversion** in a configuration with read-only replicas, you must point the indexer at the primary replica. Only a primary replica can be used for data sync scenarios.
250257
251-
#### Usage
252-
253-
To use a high water mark policy, create or update your data source like this:
258+
Change detection policies are added to data source definitions. To use this policy, create or update your data source like this:
254259

255260
```http
261+
POST https://myservice.search.windows.net/datasources?api-version=2020-06-30
262+
Content-Type: application/json
263+
api-key: admin-key
256264
{
257265
"name" : "myazuresqldatasource",
258266
"type" : "azuresql",
259267
"credentials" : { "connectionString" : "connection string" },
260268
"container" : { "name" : "table or view name" },
261269
"dataChangeDetectionPolicy" : {
262-
"@odata.type" : "#Microsoft.Azure.Search.HighWaterMarkChangeDetectionPolicy",
263-
"highWaterMarkColumnName" : "[a rowversion or last_updated column name]"
264-
}
270+
"@odata.type" : "#Microsoft.Azure.Search.HighWaterMarkChangeDetectionPolicy",
271+
"highWaterMarkColumnName" : "[a rowversion or last_updated column name]"
272+
}
265273
}
266274
```
267275

268-
> [!WARNING]
276+
> [!NOTE]
269277
> If the source table does not have an index on the high water mark column, queries used by the SQL indexer may time out. In particular, the `ORDER BY [High Water Mark Column]` clause requires an index to run efficiently when the table contains many rows.
270-
>
271-
>
272278
273279
<a name="convertHighWaterMarkToRowVersion"></a>
274280

0 commit comments

Comments
 (0)