Merge pull request #174262 from HeidiSteen/heidist-fresh

[azure search] cross-link to ADLS Gen2 sample

Author: PRMerger6

articles/search/samples-dotnet.md

@@ -8,7 +8,7 @@ author: HeidiSteen
 ms.author: heidist
 ms.service: cognitive-search
 ms.topic: conceptual
-ms.date: 01/27/2021
+ms.date: 10/01/2021
 ---
 
 # .NET (C#) code samples for Azure Cognitive Search
@@ -62,6 +62,7 @@ The following samples are also published by the Cognitive Search team, but are n
 
 | Samples | Description |
 |---------|-------------|
+| [Index Data Lake Gen2 using Azure AD](https://github.com/Azure-Samples/azure-search-dotnet-samples/blob/master/data-lake-gen2-acl-indexing/README.md) | Source code demonstrating indexer connections and indexing of Azure Data Lake Gen2 files and folders that are secured through Azure AD and role-based access controls. |
 | [azure-search-power-skills](https://github.com/Azure-Samples/azure-search-power-skills)  | Source code for consumable custom skills that you can incorporate in your won solutions.  |
 | [Knowledge Mining Solution Accelerator](/samples/azure-samples/azure-search-knowledge-mining/azure-search-knowledge-mining/) | Includes templates, support files, and analytical reports to help you prototype an end-to-end knowledge mining solution.  |
 | [Covid-19 Search App repository](https://github.com/liamca/covid19search) | Source code repository for the Cognitive Search based [Covid-19 Search App](https://covid19search.azurewebsites.net/) |

articles/search/search-howto-index-azure-data-lake-storage.md

@@ -7,7 +7,7 @@ author: markheff
 ms.author: maheff
 ms.service: cognitive-search
 ms.topic: conceptual
-ms.date: 05/17/2021
+ms.date: 10/01/2021
 ---
 
 # Index data from Azure Data Lake Storage Gen2
@@ -16,13 +16,19 @@ This article shows you how to configure an Azure Data Lake Storage Gen2 indexer
 
 Azure Data Lake Storage Gen2 is available through Azure Storage. When setting up an Azure storage account, you have the option to enable [hierarchical namespace](../storage/blobs/data-lake-storage-namespace.md). This allows the collection of content in an account to be organized into a hierarchy of directories and nested subdirectories. By enabling hierarchical namespace, you enable [Azure Data Lake Storage Gen2](../storage/blobs/data-lake-storage-introduction.md).
 
+Examples in this article use the portal and REST APIs. For examples in C#, see [Index Data Lake Gen2 using Azure AD](https://github.com/Azure-Samples/azure-search-dotnet-samples/blob/master/data-lake-gen2-acl-indexing/README.md) on GitHub.
+
 ## Supported access tiers
 
 Data Lake Storage Gen2 [access tiers](../storage/blobs/access-tiers-overview.md) include hot, cool, and archive. Only hot and cool can be accessed by indexers.
 
 ## Access control
 
-Data Lake Storage Gen2 implements an [access control model](../storage/blobs/data-lake-storage-access-control.md) that supports both Azure role-based access control (Azure RBAC) and POSIX-like access control lists (ACLs). When indexing content from Data Lake Storage Gen2, Azure Cognitive Search will not extract the Azure RBAC and ACL information from the content. As a result, this information will not be included in your Azure Cognitive Search index.
+Data Lake Storage Gen2 implements an [access control model](../storage/blobs/data-lake-storage-access-control.md) that supports both Azure role-based access control (Azure RBAC) and POSIX-like access control lists (ACLs). Access control lists are partially supported in Azure Cognitive Search scenarios:
+
++ Support for access control is enabled on indexer access to content in Data Lake Storage Gen2. For a search service that has a system or user-assigned managed identity, you can define role assignments that determine indexer access to specific files and folders in Azure Storage.
+
++ Support for document-level permissions on an index is not available. If your access controls vary the level of access on a per user basis, those permissions cannot be carried forward into a search index on your search service. All users have the same level of access to all searchable and retrievable content in the index.
 
 If maintaining access control on each document in the index is important, it is up to the application developer to implement [security trimming](./search-security-trimming-for-azure-search.md).
 
@@ -34,11 +40,11 @@ The Azure Cognitive Search blob indexer can extract text from the following docu
 
 [!INCLUDE [search-blob-data-sources](../../includes/search-blob-data-sources.md)]
 
-## Getting started with the Azure portal
+## Indexing through the Azure portal
 
 The Azure portal supports importing data from Azure Data Lake Storage Gen2. To import data from Data Lake Storage Gen2, navigate to your Azure Cognitive Search service page in the Azure portal, select **Import data**, select **Azure Data Lake Storage Gen2**, then continue to follow the Import data flow to create your data source, skillset, index, and indexer.
 
-## Getting started with the REST API
+## Indexing with the REST API
 
 The Data Lake Storage Gen2 indexer is supported by the REST API. Follow the instructions below to set up a data source, index, and indexer.
 
@@ -93,7 +99,7 @@ The SAS should have the list and read permissions on the container. For more inf
 
 ### Step 2 - Create an index
 
-The index specifies the fields in a document, attributes, and other constructs that shape the search experience. All indexers require that you specify a search index definition as the destination. The following example creates a simple index using the [Create Index (REST API)](/rest/api/searchservice/create-index). 
+The index specifies the fields in a document, attributes, and other constructs that shape the search experience. All indexers require that you specify a search index definition as the destination. The following example uses the [Create Index (REST API)](/rest/api/searchservice/create-index). 
 
 ```http
     POST https://[service name].search.windows.net/indexes?api-version=2020-06-30
@@ -117,7 +123,7 @@ You could also add fields for any blob metadata that you want in the index. The
 
 ### Step 3 - Configure and run the indexer
 
-Once the index and data source have been created, you're ready to create the indexer:
+Once the index and data source have been created, you're ready to [create the indexer](/rest/api/searchservice/create-indexer):
 
 ```http
     POST https://[service name].search.windows.net/indexers?api-version=2020-06-30
@@ -134,11 +140,7 @@ Once the index and data source have been created, you're ready to create the ind
     }
 ```
 
-This indexer runs every two hours (schedule interval is set to "PT2H"). To run an indexer every 30 minutes, set the interval to "PT30M". The shortest supported interval is 5 minutes. The schedule is optional - if omitted, an indexer runs only once when it's created. However, you can run an indexer on-demand at any time.   
-
-For more details on the Create Indexer API, check out [Create Indexer](/rest/api/searchservice/create-indexer).
-
-For more information about defining indexer schedules, see [How to schedule indexers for Azure Cognitive Search](search-howto-schedule-indexers.md).
+This indexer runs immediately, and then [on a schedule](search-howto-schedule-indexers.md) every two hours (schedule interval is set to "PT2H"). To run an indexer every 30 minutes, set the interval to "PT30M". The shortest supported interval is 5 minutes. The schedule is optional - if omitted, an indexer runs only once when it's created. However, you can run an indexer on-demand at any time.
 
 <a name="DocumentKeys"></a>
 
@@ -265,7 +267,11 @@ It's important to point out that you don't need to define fields for all of the
 
 ## How to control which blobs are indexed
 
-You can control which blobs are indexed, and which are skipped, by the blob's file type or by setting properties on the blob themselves, causing the indexer to skip over them.
+You can control which blobs are indexed, and which are skipped, by setting role assignments, the blob's file type, or by setting properties on the blob themselves, causing the indexer to skip over them.
+
+### Use access controls and role assignments
+
+Indexers that run under a system or user-assigned managed identity can have membership in either a Reader or Storage Blob Data Reader role that grants read permissions on specific files and folders.
 
 ### Include specific file extensions
 
@@ -381,6 +387,7 @@ You can also set [blob configuration properties](/rest/api/searchservice/create-
 
 ## See also
 
++ [C# Sample: Index Data Lake Gen2 using Azure AD](https://github.com/Azure-Samples/azure-search-dotnet-samples/blob/master/data-lake-gen2-acl-indexing/README.md)
 + [Indexers in Azure Cognitive Search](search-indexer-overview.md)
 + [Create an indexer](search-howto-create-indexers.md)
 + [AI enrichment over blobs overview](search-blob-ai-integration.md)

articles/search/search-howto-managed-identities-storage.md

@@ -7,7 +7,7 @@ author: markheff
 ms.author: maheff
 ms.service: cognitive-search
 ms.topic: conceptual
-ms.date: 07/02/2021
+ms.date: 10/01/2021
 ---
 
 # Set up a connection to an Azure Storage account using a managed identity
@@ -16,15 +16,18 @@ This page describes how to set up an indexer connection to an Azure storage acco
 
 You can use a system-assigned managed identity or a user-assigned managed identity (preview).
 
-Before learning more about this feature, it is recommended that you have an understanding of what an indexer is and how to set up an indexer for your data source. More information can be found at the following links:
+This article assumes familiarity with indexer concepts and configuration. If you're new to indexers, start with these links:
+
 * [Indexer overview](search-indexer-overview.md)
 * [Azure Blob indexer](search-howto-indexing-azure-blob-storage.md)
 * [Azure Data Lake Storage Gen2 indexer](search-howto-index-azure-data-lake-storage.md)
 * [Azure Table indexer](search-howto-indexing-azure-tables.md)
 
 ## 1 - Set up a managed identity
 
-Set up the [managed identity](../active-directory/managed-identities-azure-resources/overview.md) using one of the following options.
+Set up the [managed identity](../active-directory/managed-identities-azure-resources/overview.md) for an Azure Cognitive Search service using one of the following options. 
+
+The search service must be Basic tier or above.
 
 ### Option 1 - Turn on system-assigned managed identity
 
@@ -35,26 +38,32 @@ When a system-assigned managed identity is enabled, Azure creates an identity fo
 After selecting **Save** you will see an Object ID that has been assigned to your search service.
 
 ![Object ID](./media/search-managed-identities/system-assigned-identity-object-id.png "Object ID")
- 
+
 ### Option 2 - Assign a user-assigned managed identity to the search service (preview)
 
 If you don't already have a user-assigned managed identity created, you'll need to create one. A user-assigned managed identity is a resource on Azure.
 
 1. Sign into the [Azure portal](https://portal.azure.com/).
+
 1. Select **+ Create a resource**.
+
 1. In the "Search services and marketplace" search bar, search for "User Assigned Managed Identity" and then select **Create**.
+
 1. Give the identity a descriptive name.
 
 Next, assign the user-assigned managed identity to the search service. This can be done using the [2021-04-01-preview management API](/rest/api/searchmanagement/2021-04-01-preview/services/create-or-update).
 
 The identity property takes a type and one or more fully-qualified user-assigned identities:
 
-* **type** is the type of identity. Valid values are "SystemAssigned", "UserAssigned", or "SystemAssigned, UserAssigned" if you want to use both. A value of "None" will clear any previously assigned identities from the search service.
-* **userAssignedIdentities** includes the details of the user assigned managed identity.
-    * User-assigned managed identity format: 
-        * /subscriptions/**subscription ID**/resourcegroups/**resource group name**/providers/Microsoft.ManagedIdentity/userAssignedIdentities/**name of managed identity**
+* **type** is the type of identity. Valid values are "SystemAssigned", "UserAssigned", or "SystemAssigned, UserAssigned" for both. A value of "None" will clear any previously assigned identities from the search service.
+
+* **userAssignedIdentities** includes the details of the user assigned managed identity. The format is:
+
+  ```bash
+    /subscriptions/<your-subscription-ID>/resourcegroups/<your-resource-group-name>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<your-managed-identity-name>
+  ```
 
-Example of how to assign a user-assigned managed identity to a search service:
+Example of a user-assigned managed identity assignment:
 
 ```http
 PUT https://management.azure.com/subscriptions/[subscription ID]/resourceGroups/[resource group name]/providers/Microsoft.Search/searchServices/[search service name]?api-version=2021-04-01-preview
@@ -81,20 +90,25 @@ Content-Type: application/json
 
 ## 2 - Add a role assignment
 
-In this step you will either give your Azure Cognitive Search service or user-assigned managed identity permission to read data from your storage account.
+In this step, you will either give your Azure Cognitive Search service or user-assigned managed identity permission to read data from your storage account.
 
 1. In the Azure portal, navigate to the Storage account that contains the data that you would like to index.
+
 2. Select **Access control (IAM)**
+
 3. Select **Add** then **Add role assignment**
 
     ![Add role assignment](./media/search-managed-identities/add-role-assignment-storage.png "Add role assignment")
 
 4. Select the appropriate role(s) based on the storage account type that you would like to index:
-    1. Azure Blob Storage requires that you add your search service to the **Storage Blob Data Reader** role.
-    1. Azure Data Lake Storage Gen2 requires that you add your search service to the **Storage Blob Data Reader** role.
-    1. Azure Table Storage requires that you add your search service to the **Reader and Data Access** role.
-5.	Leave **Assign access to** as **Azure AD user, group or service principal**
-6.	If you're using a system-assigned managed identity, search for your search service, then select it. If you're using a user-assigned managed identity, search for the name of the user-assigned managed identity, then select it. Select **Save**.
+
+    * Azure Blob Storage requires that you add your search service to the **Storage Blob Data Reader** role.
+    * Azure Data Lake Storage Gen2 requires that you add your search service to the **Storage Blob Data Reader** role.
+    * Azure Table Storage requires that you add your search service to the **Reader and Data Access** role.
+
+5. Leave **Assign access to** as **Azure AD user, group or service principal**
+
+6. If you're using a system-assigned managed identity, search for your search service, then select it. If you're using a user-assigned managed identity, search for the name of the user-assigned managed identity, then select it. Select **Save**.
 
     Example for Azure Blob Storage and Azure Data Lake Storage Gen2 using a system-assigned managed identity:
 
@@ -104,6 +118,8 @@ In this step you will either give your Azure Cognitive Search service or user-as
 
     ![Add reader and data access role assignment](./media/search-managed-identities/add-role-assignment-reader-and-data-access.png "Add reader and data access role assignment")
 
+For a code examples in C#, see [Index Data Lake Gen2 using Azure AD](https://github.com/Azure-Samples/azure-search-dotnet-samples/blob/master/data-lake-gen2-acl-indexing/README.md) on GitHub.
+
 ## 3 - Create the data source
 
 Create the data source and provide either a system-assigned managed identity or a user-assigned managed identity (preview). Note that you are no longer using the Management REST API in the below steps.
@@ -237,14 +253,13 @@ For more details on the Create Indexer API, check out [Create Indexer](/rest/api
 
 For more information about defining indexer schedules see [How to schedule indexers for Azure Cognitive Search](search-howto-schedule-indexers.md).
 
-## Accessing secure data in storage accounts
+## Accessing network secured data in storage accounts
 
 Azure storage accounts can be further secured using firewalls and virtual networks. If you want to index content from a blob storage account or Data Lake Gen2 storage account that is secured using a firewall or virtual network, follow the instructions for [Accessing data in storage accounts securely via trusted service exception](search-indexer-howto-access-trusted-service-exception.md).
 
 ## See also
 
-Learn more about Azure Storage indexers:
-
 * [Azure Blob indexer](search-howto-indexing-azure-blob-storage.md)
 * [Azure Data Lake Storage Gen2 indexer](search-howto-index-azure-data-lake-storage.md)
-* [Azure Table indexer](search-howto-indexing-azure-tables.md)
+* [Azure Table indexer](search-howto-indexing-azure-tables.md)
+* [C# Example: Index Data Lake Gen2 using Azure AD (GitHub)](https://github.com/Azure-Samples/azure-search-dotnet-samples/blob/master/data-lake-gen2-acl-indexing/README.md)