Skip to content

Commit 2461dbc

Browse files
authored
Merge pull request #189979 from HeidiSteen/heidist-work
Added H2 check status to indexers
2 parents d15ddc5 + 3a70774 commit 2461dbc

9 files changed

+335
-13
lines changed

articles/search/search-file-storage-integration.md

Lines changed: 47 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -130,7 +130,7 @@ In the [search index](search-what-is-an-index.md), add fields to accept the cont
130130
+ **metadata_storage_content_md5** (`Edm.String`) - MD5 hash of the file content, if available.
131131
+ **metadata_storage_sas_token** (`Edm.String`) - A temporary SAS token that can be used by [custom skills](cognitive-search-custom-skill-interface.md) to get access to the file. This token shouldn't be stored for later use as it might expire.
132132

133-
## Configure the file indexer
133+
## Configure and run the file indexer
134134

135135
Indexer configuration specifies the inputs, parameters, and properties controlling run time behaviors. Under "configuration", you can specify which files are indexed by file type or by properties on the files themselves.
136136

@@ -167,6 +167,52 @@ Indexer configuration specifies the inputs, parameters, and properties controlli
167167

168168
1. See [Create an indexer](search-howto-create-indexers.md) for more information about other properties.
169169

170+
## Check indexer status
171+
172+
To monitor the indexer status and execution history, send a [Get Indexer Status](/rest/api/searchservice/get-indexer-status) request:
173+
174+
```http
175+
GET https://myservice.search.windows.net/indexers/myindexer/status?api-version=2020-06-30
176+
Content-Type: application/json
177+
api-key: [admin key]
178+
```
179+
180+
The response includes status and the number of items processed. It should look similar to the following example:
181+
182+
```json
183+
{
184+
"status":"running",
185+
"lastResult": {
186+
"status":"success",
187+
"errorMessage":null,
188+
"startTime":"2022-02-21T00:23:24.957Z",
189+
"endTime":"2022-02-21T00:36:47.752Z",
190+
"errors":[],
191+
"itemsProcessed":1599501,
192+
"itemsFailed":0,
193+
"initialTrackingState":null,
194+
"finalTrackingState":null
195+
},
196+
"executionHistory":
197+
[
198+
{
199+
"status":"success",
200+
"errorMessage":null,
201+
"startTime":"2022-02-21T00:23:24.957Z",
202+
"endTime":"2022-02-21T00:36:47.752Z",
203+
"errors":[],
204+
"itemsProcessed":1599501,
205+
"itemsFailed":0,
206+
"initialTrackingState":null,
207+
"finalTrackingState":null
208+
},
209+
... earlier history items
210+
]
211+
}
212+
```
213+
214+
Execution history contains up to 50 of the most recently completed executions, which are sorted in the reverse chronological order so that the latest execution comes first.
215+
170216
## Next steps
171217

172218
You can now [run the indexer](search-howto-run-reset-indexers.md), [monitor status](search-howto-monitor-indexers.md), or [schedule indexer execution](search-howto-schedule-indexers.md). The following articles apply to indexers that pull content from Azure Storage:

articles/search/search-howto-connecting-azure-sql-database-to-azure-search-using-indexers.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ ms.date: 02/28/2022
1313

1414
# Index data from Azure SQL
1515

16-
In this article, learn how to configure an [**indexer**](search-indexer-overview.md) that imports content from Azure SQL and makes it searchable in Azure Cognitive Search.
16+
In this article, learn how to configure an [**indexer**](search-indexer-overview.md) that imports content from Azure SQL Database or an Azure SQL managed instance and makes it searchable in Azure Cognitive Search.
1717

1818
This article supplements [**Create an indexer**](search-howto-create-indexers.md) with information that's specific to Azure SQL. It uses the REST APIs to demonstrate a three-part workflow common to all indexers: create a data source, create an index, create an indexer. Data extraction occurs when you submit the Create Indexer request.
1919

articles/search/search-howto-create-indexers.md

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -216,25 +216,25 @@ There are several ways to run an indexer:
216216

217217
Scheduled execution is usually implemented when you have a need for incremental indexing so that you can pick up the latest changes. As such, scheduling has a dependency on change detection.
218218

219-
## Change detection and internal state
219+
## Check results
220220

221-
Change detection logic is a capability that's built into source platforms. If your data source support change detection, an indexer can detect changes in the underlying data and only process new or updated documents on each indexer run, leaving unchanged content as-is. If indexer execution history says that a run was successful with `0/0` documents processed, it means that the indexer didn't find any new or changed rows or blobs in the underlying data source.
221+
[Monitor indexer status](search-howto-monitor-indexers.md) to check for status. Successful execution can still include warning and notifications. Be sure to check both successful and failed status notifications for details about the job.
222222

223-
How an indexer supports change detection varies by data source:
223+
For content verification, [run queries](search-query-create.md) on the populated index that return entire documents or selected fields.
224224

225-
+ Azure Blob Storage, Azure Table Storage, and Azure Data Lake Storage Gen2 stamp each blob or row update with a date and time. The various indexers use this information to determine which documents to update in the index. Built-in change detection means that an indexer can recognize new and updated documents automatically.
225+
## Change detection and internal state
226226

227-
+ Azure SQL and Cosmos DB provide change detection features in their platforms. You can specify the change detection policy in your data source definition.
227+
If your data source supports change detection, an indexer can detect underlying changes in the data and process just the new or updated documents on each indexer run, leaving unchanged content as-is. If indexer execution history says that a run was successful with `0/0` documents processed, it means that the indexer didn't find any new or changed rows or blobs in the underlying data source.
228228

229-
For large indexing loads, an indexer also keeps track of the last document it processed through an internal "high water mark". The marker is never exposed in the API, but internally the indexer keeps track of where it stopped. When indexing resumes, either through a scheduled run or an on-demand invocation, the indexer references the high water mark so that it can pick up where it left off.
229+
Change detection logic is built into the data platforms. How an indexer supports change detection varies by data source:
230230

231-
If you need to clear the high water mark to re-index in full, you can use [Reset Indexer](/rest/api/searchservice/reset-indexer). For more selective re-indexing, use [Reset Skills](/rest/api/searchservice/preview-api/reset-skills) or [Reset Documents](/rest/api/searchservice/preview-api/reset-documents). Through the reset APIs, you can clear internal state, and also flush the cache if you enabled [incremental enrichment](search-howto-incremental-index.md). For more background and comparison of each reset option, see [Run or reset indexers, skills, and documents](search-howto-run-reset-indexers.md).
231+
+ Azure Storage has built-in change detection, which means an indexer can recognize new and updated documents automatically.. Blob Storage, Azure Table Storage, and Azure Data Lake Storage Gen2 stamp each blob or row update with a date and time. An indexer can use this information to determine which documents to update in the index.
232232

233-
## Check results
233+
+ Azure SQL and Cosmos DB provide optional change detection features in their platforms. You can specify the change detection policy in your data source definition.
234234

235-
[Monitor indexer status](search-howto-monitor-indexers.md) to check for status. Successful execution can still include warning and notifications. Be sure to check both successful and failed status notifications for details about the job.
235+
For large indexing loads, an indexer also keeps track of the last document it processed through an internal "high water mark". The marker is never exposed in the API, but internally the indexer keeps track of where it stopped. When indexing resumes, either through a scheduled run or an on-demand invocation, the indexer references the high water mark so that it can pick up where it left off.
236236

237-
For content verification, [run queries](search-query-create.md) on the populated index that return entire documents or selected fields.
237+
If you need to clear the high water mark to re-index in full, you can use [Reset Indexer](/rest/api/searchservice/reset-indexer). For more selective re-indexing, use [Reset Skills](/rest/api/searchservice/preview-api/reset-skills) or [Reset Documents](/rest/api/searchservice/preview-api/reset-documents). Through the reset APIs, you can clear internal state, and also flush the cache if you enabled [incremental enrichment](search-howto-incremental-index.md). For more background and comparison of each reset option, see [Run or reset indexers, skills, and documents](search-howto-run-reset-indexers.md).
238238

239239
## Next steps
240240

articles/search/search-howto-index-azure-data-lake-storage.md

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -185,6 +185,52 @@ Indexer configuration specifies the inputs, parameters, and properties controlli
185185

186186
For the full list of parameter descriptions, see [Blob configuration parameters](/rest/api/searchservice/create-indexer#blob-configuration-parameters) in the REST API.
187187

188+
## Check indexer status
189+
190+
To monitor the indexer status and execution history, send a [Get Indexer Status](/rest/api/searchservice/get-indexer-status) request:
191+
192+
```http
193+
GET https://myservice.search.windows.net/indexers/myindexer/status?api-version=2020-06-30
194+
Content-Type: application/json
195+
api-key: [admin key]
196+
```
197+
198+
The response includes status and the number of items processed. It should look similar to the following example:
199+
200+
```json
201+
{
202+
"status":"running",
203+
"lastResult": {
204+
"status":"success",
205+
"errorMessage":null,
206+
"startTime":"2022-02-21T00:23:24.957Z",
207+
"endTime":"2022-02-21T00:36:47.752Z",
208+
"errors":[],
209+
"itemsProcessed":1599501,
210+
"itemsFailed":0,
211+
"initialTrackingState":null,
212+
"finalTrackingState":null
213+
},
214+
"executionHistory":
215+
[
216+
{
217+
"status":"success",
218+
"errorMessage":null,
219+
"startTime":"2022-02-21T00:23:24.957Z",
220+
"endTime":"2022-02-21T00:36:47.752Z",
221+
"errors":[],
222+
"itemsProcessed":1599501,
223+
"itemsFailed":0,
224+
"initialTrackingState":null,
225+
"finalTrackingState":null
226+
},
227+
... earlier history items
228+
]
229+
}
230+
```
231+
232+
Execution history contains up to 50 of the most recently completed executions, which are sorted in the reverse chronological order so that the latest execution comes first.
233+
188234
## How blobs are indexed
189235

190236
By default, most blobs are indexed as a single search document in the index, including blobs with structured content, such as JSON or CSV, which are indexed as a single chunk of text. However, for JSON or CSV documents that have an internal structure (delimiters), you can assign parsing modes to generate individual search documents for each line or element:

articles/search/search-howto-index-cosmosdb-gremlin.md

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -190,6 +190,52 @@ Indexer configuration specifies the inputs, parameters, and properties controlli
190190
191191
1. See [Create an indexer](search-howto-create-indexers.md) for more information about other properties.
192192
193+
## Check indexer status
194+
195+
To monitor the indexer status and execution history, send a [Get Indexer Status](/rest/api/searchservice/get-indexer-status) request:
196+
197+
```http
198+
GET https://myservice.search.windows.net/indexers/myindexer/status?api-version=2020-06-30
199+
Content-Type: application/json
200+
api-key: [admin key]
201+
```
202+
203+
The response includes status and the number of items processed. It should look similar to the following example:
204+
205+
```json
206+
{
207+
"status":"running",
208+
"lastResult": {
209+
"status":"success",
210+
"errorMessage":null,
211+
"startTime":"2022-02-21T00:23:24.957Z",
212+
"endTime":"2022-02-21T00:36:47.752Z",
213+
"errors":[],
214+
"itemsProcessed":1599501,
215+
"itemsFailed":0,
216+
"initialTrackingState":null,
217+
"finalTrackingState":null
218+
},
219+
"executionHistory":
220+
[
221+
{
222+
"status":"success",
223+
"errorMessage":null,
224+
"startTime":"2022-02-21T00:23:24.957Z",
225+
"endTime":"2022-02-21T00:36:47.752Z",
226+
"errors":[],
227+
"itemsProcessed":1599501,
228+
"itemsFailed":0,
229+
"initialTrackingState":null,
230+
"finalTrackingState":null
231+
},
232+
... earlier history items
233+
]
234+
}
235+
```
236+
237+
Execution history contains up to 50 of the most recently completed executions, which are sorted in the reverse chronological order so that the latest execution comes first.
238+
193239
<a name="DataChangeDetectionPolicy"></a>
194240

195241
## Indexing changed documents

articles/search/search-howto-index-cosmosdb-mongodb.md

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -173,6 +173,52 @@ Indexer configuration specifies the inputs, parameters, and properties controlli
173173
174174
1. See [Create an indexer](search-howto-create-indexers.md) for more information about other properties.
175175
176+
## Check indexer status
177+
178+
To monitor the indexer status and execution history, send a [Get Indexer Status](/rest/api/searchservice/get-indexer-status) request:
179+
180+
```http
181+
GET https://myservice.search.windows.net/indexers/myindexer/status?api-version=2020-06-30
182+
Content-Type: application/json
183+
api-key: [admin key]
184+
```
185+
186+
The response includes status and the number of items processed. It should look similar to the following example:
187+
188+
```json
189+
{
190+
"status":"running",
191+
"lastResult": {
192+
"status":"success",
193+
"errorMessage":null,
194+
"startTime":"2022-02-21T00:23:24.957Z",
195+
"endTime":"2022-02-21T00:36:47.752Z",
196+
"errors":[],
197+
"itemsProcessed":1599501,
198+
"itemsFailed":0,
199+
"initialTrackingState":null,
200+
"finalTrackingState":null
201+
},
202+
"executionHistory":
203+
[
204+
{
205+
"status":"success",
206+
"errorMessage":null,
207+
"startTime":"2022-02-21T00:23:24.957Z",
208+
"endTime":"2022-02-21T00:36:47.752Z",
209+
"errors":[],
210+
"itemsProcessed":1599501,
211+
"itemsFailed":0,
212+
"initialTrackingState":null,
213+
"finalTrackingState":null
214+
},
215+
... earlier history items
216+
]
217+
}
218+
```
219+
220+
Execution history contains up to 50 of the most recently completed executions, which are sorted in the reverse chronological order so that the latest execution comes first.
221+
176222
<a name="DataChangeDetectionPolicy"></a>
177223

178224
## Indexing changed documents

articles/search/search-howto-index-mysql.md

Lines changed: 47 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -103,7 +103,7 @@ In a [search index](search-what-is-an-index.md), add search index fields that co
103103

104104
If the primary key in the source table matches the document key (in this case, "ID"), the indexer will import the primary key as the document key.
105105

106-
## Configure the MySQL indexer
106+
## Configure and run the MySQL indexer
107107

108108
Once the index and data source have been created, you're ready to create the indexer.
109109

@@ -151,6 +151,52 @@ api-key: [admin-key]
151151
}
152152
```
153153

154+
## Check indexer status
155+
156+
To monitor the indexer status and execution history, send a [Get Indexer Status](/rest/api/searchservice/get-indexer-status) request:
157+
158+
```http
159+
GET https://myservice.search.windows.net/indexers/myindexer/status?api-version=2020-06-30
160+
Content-Type: application/json
161+
api-key: [admin key]
162+
```
163+
164+
The response includes status and the number of items processed. It should look similar to the following example:
165+
166+
```json
167+
{
168+
"status":"running",
169+
"lastResult": {
170+
"status":"success",
171+
"errorMessage":null,
172+
"startTime":"2022-02-21T00:23:24.957Z",
173+
"endTime":"2022-02-21T00:36:47.752Z",
174+
"errors":[],
175+
"itemsProcessed":1599501,
176+
"itemsFailed":0,
177+
"initialTrackingState":null,
178+
"finalTrackingState":null
179+
},
180+
"executionHistory":
181+
[
182+
{
183+
"status":"success",
184+
"errorMessage":null,
185+
"startTime":"2022-02-21T00:23:24.957Z",
186+
"endTime":"2022-02-21T00:36:47.752Z",
187+
"errors":[],
188+
"itemsProcessed":1599501,
189+
"itemsFailed":0,
190+
"initialTrackingState":null,
191+
"finalTrackingState":null
192+
},
193+
... earlier history items
194+
]
195+
}
196+
```
197+
198+
Execution history contains up to 50 of the most recently completed executions, which are sorted in the reverse chronological order so that the latest execution comes first.
199+
154200
## Capture new, changed, and deleted rows
155201

156202
If your data source meets the requirements for change and deletion detection, the indexer can incrementally index the changes in your data source since the last indexer job, which means you can avoid having to re-index the entire table or view every time an indexer runs.

articles/search/search-howto-indexing-azure-blob-storage.md

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -179,6 +179,52 @@ Indexer configuration specifies the inputs, parameters, and properties controlli
179179

180180
For the full list of parameter descriptions, see [Blob configuration parameters](/rest/api/searchservice/create-indexer#blob-configuration-parameters) in the REST API.
181181

182+
## Check indexer status
183+
184+
To monitor the indexer status and execution history, send a [Get Indexer Status](/rest/api/searchservice/get-indexer-status) request:
185+
186+
```http
187+
GET https://myservice.search.windows.net/indexers/myindexer/status?api-version=2020-06-30
188+
Content-Type: application/json
189+
api-key: [admin key]
190+
```
191+
192+
The response includes status and the number of items processed. It should look similar to the following example:
193+
194+
```json
195+
{
196+
"status":"running",
197+
"lastResult": {
198+
"status":"success",
199+
"errorMessage":null,
200+
"startTime":"2022-02-21T00:23:24.957Z",
201+
"endTime":"2022-02-21T00:36:47.752Z",
202+
"errors":[],
203+
"itemsProcessed":1599501,
204+
"itemsFailed":0,
205+
"initialTrackingState":null,
206+
"finalTrackingState":null
207+
},
208+
"executionHistory":
209+
[
210+
{
211+
"status":"success",
212+
"errorMessage":null,
213+
"startTime":"2022-02-21T00:23:24.957Z",
214+
"endTime":"2022-02-21T00:36:47.752Z",
215+
"errors":[],
216+
"itemsProcessed":1599501,
217+
"itemsFailed":0,
218+
"initialTrackingState":null,
219+
"finalTrackingState":null
220+
},
221+
... earlier history items
222+
]
223+
}
224+
```
225+
226+
Execution history contains up to 50 of the most recently completed executions, which are sorted in the reverse chronological order so that the latest execution comes first.
227+
182228
## How blobs are indexed
183229

184230
By default, most blobs are indexed as a single search document in the index, including blobs with structured content, such as JSON or CSV, which are indexed as a single chunk of text. However, for JSON or CSV documents that have an internal structure (delimiters), you can assign parsing modes to generate individual search documents for each line or element:

0 commit comments

Comments
 (0)