You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/search/search-howto-connecting-azure-sql-database-to-azure-search-using-indexers.md
+17-17Lines changed: 17 additions & 17 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -24,15 +24,15 @@ This article also provides:
24
24
+ A frequently-asked-questions (FAQ) section for answers to questions about feature compatibility.
25
25
26
26
> [!NOTE]
27
-
> [Always Encrypted](/sql/relational-databases/security/encryption/always-encrypted-database-engine) columns are not currently supported by Cognitive Search indexers.
27
+
> [Always Encrypted](/sql/relational-databases/security/encryption/always-encrypted-database-engine) columns aren't currently supported by Cognitive Search indexers.
28
28
29
29
## Prerequisites
30
30
31
31
+ An [Azure SQL database](/azure/azure-sql/database/sql-database-paas-overview) with data in a single table or view.
32
32
33
33
Use a table if your data is over 100,000 rows or if you need [incremental indexing](#CaptureChangedRows) using SQL's native change detection capabilities.
34
34
35
-
Use a view if you need to consolidate data from multiple tables. Large views are not ideal for SQL indexer. A workaround is to create a new single table just for ingestion into your Cognitive Search index. You'll be able to use SQL integrated change tracking, which is easier to implement than High Water Mark.
35
+
Use a view if you need to consolidate data from multiple tables. Large views aren't ideal for SQL indexer. A workaround is to create a new single table just for ingestion into your Cognitive Search index. You'll be able to use SQL integrated change tracking, which is easier to implement than High Water Mark.
36
36
37
37
+ Read permissions. Azure Cognitive Search supports SQL Server authentication, where the user name and password are provided on the connection string. Alternatively, you can [set up a managed identity and use Azure roles](search-howto-managed-identities-sql.md).
38
38
@@ -80,7 +80,7 @@ The data source definition specifies the data to index, credentials, and policie
80
80
81
81
+ You can get a full access connection string from the [Azure portal](https://portal.azure.com). Use the `ADO.NET connection string` option. Set the user name and password.
82
82
83
-
+ Alternatively, you can specify a managed identity connection string that does not include database secrets with the following format: `Initial Catalog|Database=<your database name>;ResourceId=/subscriptions/<your subscription ID>/resourceGroups/<your resource group name>/providers/Microsoft.Sql/servers/<your SQL Server name>/;Connection Timeout=connection timeout length;`.
83
+
+ Alternatively, you can specify a managed identity connection string that doesn't include database secrets with the following format: `Initial Catalog|Database=<your database name>;ResourceId=/subscriptions/<your subscription ID>/resourceGroups/<your resource group name>/providers/Microsoft.Sql/servers/<your SQL Server name>/;Connection Timeout=connection timeout length;`.
84
84
85
85
For more information, see [Connect to Azure SQL Database indexer using a managed identity](search-howto-managed-identities-sql.md).
86
86
@@ -129,12 +129,12 @@ In a [search index](search-what-is-an-index.md), add fields that correspond to t
| smallmoney, money decimal numeric |Edm.String |Azure Cognitive Search does not support converting decimal types into Edm.Double because doing so would lose precision |
133
-
| char, nchar, varchar, nvarchar |Edm.String<br/>Collection(Edm.String) |A SQL string can be used to populate a Collection(Edm.String) field if the string represents a JSON array of strings: `["red", "white", "blue"]` |
132
+
| smallmoney, money decimal numeric |Edm.String |Azure Cognitive Search doesn't support converting decimal types into `Edm.Double` because doing so would lose precision |
133
+
| char, nchar, varchar, nvarchar |Edm.String<br/>Collection(Edm.String) |A SQL string can be used to populate a Collection(`Edm.String`) field if the string represents a JSON array of strings: `["red", "white", "blue"]` |
@@ -237,7 +237,7 @@ If your SQL database supports [change tracking](/sql/relational-databases/track-
237
237
238
238
To enable incremental indexing, set the "dataChangeDetectionPolicy" property in your data source definition. This property tells the indexer which change tracking mechanism is used on your table or view.
239
239
240
-
For Azure SQL indexers, there two change detection policies:
240
+
For Azure SQL indexers, there are two change detection policies:
241
241
242
242
+ "SqlIntegratedChangeTrackingPolicy" (applies to tables only)
243
243
@@ -271,7 +271,7 @@ api-key: admin-key
271
271
}
272
272
```
273
273
274
-
When using SQL integrated change tracking policy, do not specify a separate data deletion detection policy. The SQL integrated change tracking policy has built-in support for identifying deleted rows. However, for the deleted rows to be detected automatically, the document key in your search index must be the same as the primary key in the SQL table.
274
+
When using SQL integrated change tracking policy, don't specify a separate data deletion detection policy. The SQL integrated change tracking policy has built-in support for identifying deleted rows. However, for the deleted rows to be detected automatically, the document key in your search index must be the same as the primary key in the SQL table.
275
275
276
276
> [!NOTE]
277
277
> When using [TRUNCATE TABLE](/sql/t-sql/statements/truncate-table-transact-sql) to remove a large number of rows from a SQL table, the indexer needs to be [reset](/rest/api/searchservice/reset-indexer) to reset the change tracking state to pick up row deletions.
@@ -290,7 +290,7 @@ The high water mark column must meet the following requirements:
290
290
+ Queries with the following WHERE and ORDER BY clauses can be executed efficiently: `WHERE [High Water Mark Column] > [Current High Water Mark Value] ORDER BY [High Water Mark Column]`
291
291
292
292
> [!NOTE]
293
-
> We strongly recommend using the [rowversion](/sql/t-sql/data-types/rowversion-transact-sql) data type for the high water mark column. If any other data type is used, change tracking is not guaranteed to capture all changes in the presence of transactions executing concurrently with an indexer query. When using **rowversion** in a configuration with read-only replicas, you must point the indexer at the primary replica. Only a primary replica can be used for data sync scenarios.
293
+
> We strongly recommend using the [rowversion](/sql/t-sql/data-types/rowversion-transact-sql) data type for the high water mark column. If any other data type is used, change tracking isn't guaranteed to capture all changes in the presence of transactions executing concurrently with an indexer query. When using **rowversion** in a configuration with read-only replicas, you must point the indexer at the primary replica. Only a primary replica can be used for data sync scenarios.
294
294
295
295
Change detection policies are added to data source definitions. To use this policy, create or update your data source like this:
296
296
@@ -311,7 +311,7 @@ api-key: admin-key
311
311
```
312
312
313
313
> [!NOTE]
314
-
> If the source table does not have an index on the high water mark column, queries used by the SQL indexer may time out. In particular, the `ORDER BY [High Water Mark Column]` clause requires an index to run efficiently when the table contains many rows.
314
+
> If the source table doesn't have an index on the high water mark column, queries used by the SQL indexer may time out. In particular, the `ORDER BY [High Water Mark Column]` clause requires an index to run efficiently when the table contains many rows.
315
315
316
316
<aname="convertHighWaterMarkToRowVersion"></a>
317
317
@@ -351,7 +351,7 @@ If you encounter timeout errors, set the `queryTimeout` indexer configuration se
351
351
352
352
##### disableOrderByHighWaterMarkColumn
353
353
354
-
You can also disable the `ORDER BY [High Water Mark Column]` clause. However, this is not recommended because if the indexer execution is interrupted by an error, the indexer has to re-process all rows if it runs later, even if the indexer has already processed almost all the rows at the time it was interrupted. To disable the `ORDER BY` clause, use the `disableOrderByHighWaterMarkColumn` setting in the indexer definition:
354
+
You can also disable the `ORDER BY [High Water Mark Column]` clause. However, this isn't recommended because if the indexer execution is interrupted by an error, the indexer has to re-process all rows if it runs later, even if the indexer has already processed almost all the rows at the time it was interrupted. To disable the `ORDER BY` clause, use the `disableOrderByHighWaterMarkColumn` setting in the indexer definition:
355
355
356
356
```http
357
357
{
@@ -382,13 +382,13 @@ When using the soft-delete technique, you can specify the soft delete policy as
382
382
383
383
The **softDeleteMarkerValue** must be a string in the JSON representation of your data source. Use the string representation of your actual value. For example, if you have an integer column where deleted rows are marked with the value 1, use `"1"`. If you have a BIT column where deleted rows are marked with the Boolean true value, use the string literal `"True"` or `"true"`, the case doesn't matter.
384
384
385
-
If you are setting up a soft delete policy from the Azure portal, don't add quotes around the soft delete marker value. The field contents are already understood as a string and will be translated automatically into a JSON string for you. In the examples above, simply type `1`, `True` or `true` into the portal's field.
385
+
If you're setting up a soft delete policy from the Azure portal, don't add quotes around the soft delete marker value. The field contents are already understood as a string and will be translated automatically into a JSON string for you. In the examples above, simply type `1`, `True` or `true` into the portal's field.
386
386
387
387
## FAQ
388
388
389
389
**Q: Can I index Always Encrypted columns?**
390
390
391
-
No. [Always Encrypted](/sql/relational-databases/security/encryption/always-encrypted-database-engine) columns are not currently supported by Cognitive Search indexers.
391
+
No. [Always Encrypted](/sql/relational-databases/security/encryption/always-encrypted-database-engine) columns aren't currently supported by Cognitive Search indexers.
392
392
393
393
**Q: Can I use Azure SQL indexer with SQL databases running on IaaS VMs in Azure?**
394
394
@@ -404,13 +404,13 @@ It depends. For full indexing of a table or view, you can use a secondary replic
404
404
405
405
For incremental indexing, Azure Cognitive Search supports two change detection policies: SQL integrated change tracking and High Water Mark.
406
406
407
-
On read-only replicas, SQL Database does not support integrated change tracking. Therefore, you must use High Water Mark policy.
407
+
On read-only replicas, SQL Database doesn't support integrated change tracking. Therefore, you must use High Water Mark policy.
408
408
409
-
Our standard recommendation is to use the rowversion data type for the high water mark column. However, using rowversion relies on the `MIN_ACTIVE_ROWVERSION` function, which is not supported on read-only replicas. Therefore, you must point the indexer to a primary replica if you are using rowversion.
409
+
Our standard recommendation is to use the rowversion data type for the high water mark column. However, using rowversion relies on the `MIN_ACTIVE_ROWVERSION` function, which isn't supported on read-only replicas. Therefore, you must point the indexer to a primary replica if you're using rowversion.
410
410
411
-
If you attempt to use rowversion on a read-only replica, you will see the following error:
411
+
If you attempt to use rowversion on a read-only replica, you'll see the following error:
412
412
413
-
"Using a rowversion column for change tracking is not supported on secondary (read-only) availability replicas. Please update the datasource and specify a connection to the primary availability replica.Current database 'Updateability' property is 'READ_ONLY'".
413
+
"Using a rowversion column for change tracking isn't supported on secondary (read-only) availability replicas. Please update the datasource and specify a connection to the primary availability replica.Current database 'Updateability' property is 'READ_ONLY'".
414
414
415
415
**Q: Can I use an alternative, non-rowversion column for high water mark change tracking?**
0 commit comments