reverted security-trimming-aad article, we're archiving it instead

Author: HeidiSteen

articles/search/TOC.yml

@@ -466,9 +466,9 @@
         href: search-security-get-encryption-keys.md
     - name: Document-level security
       items:
-      - name: Security filters
+      - name: Security filter pattern
         href: search-security-trimming-for-azure-search.md
-      - name: Security filters with Microsoft Entra ID
+      - name: Set up a security filter using Microsoft Entra ID
         href: search-security-trimming-for-azure-search-with-aad.md
   - name: Development
     items:

articles/search/search-security-trimming-for-azure-search.md

@@ -1,5 +1,5 @@
 ---
-title: Security filters for trimming results
+title: Security filter pattern
 titleSuffix: Azure AI Search
 description: Learn how to implement security privileges at the document level for Azure AI Search search results, using security filters and user identities.
 
@@ -10,48 +10,50 @@ ms.service: cognitive-search
 ms.custom:
   - ignite-2023
 ms.topic: how-to
-ms.date: 01/10/2024
+ms.date: 06/20/2024
 ---
 
 # Security filters for trimming results in Azure AI Search
 
-Azure AI Search doesn't provide document-level permissions and can't vary search results from within the same index by user permissions. As a workaround, you can create a filter that trims search results based on a string containing a group or user identity.
+Azure AI Search doesn't provide native document-level permissions and can't vary search results from within the same index by user permissions. As a workaround, you can create a filter that trims search results based on a string containing a group or user identity.
 
-This article describes a pattern for security filtering that includes following steps:
+This article describes a pattern for security filtering having the following steps:
 
 > [!div class="checklist"]
 > * Assemble source documents with the required content
 > * Create a field for the principal identifiers 
 > * Push the documents to the search index for indexing
 > * Query the index with the `search.in` filter function
 
+It concludes with links to demos and examples that provide hands-on learning. We recommend reviewing this article first to understand the pattern.
+
 ## About the security filter pattern
 
-Although Azure AI Search doesn't integrate with security subsystems for access to content within an index, many customers who have document-level security requirements have found that filters can meet their needs.
+Although Azure AI Search doesn't integrate with security subsystems for access to content within an index, many customers who have document-level security requirements find that filters can meet their needs.
 
-In Azure AI Search, a security filter is a regular OData filter that includes or excludes a search result based on a matching value, except that in a security filter, the criteria is a string consisting of a security principal. There's no authentication or authorization through the security principal. The principal is just a string, used in a filter expression, to include or exclude a document from the search results.
+In Azure AI Search, a security filter is a regular OData filter that includes or excludes a search result based on a string consisting of a security principal. There's no authentication or authorization through the security principal. The principal is just a string, used in a filter expression, to include or exclude a document from the search results.
 
 There are several ways to achieve security filtering. One way is through a complicated disjunction of equality expressions: for example, `Id eq 'id1' or Id eq 'id2'`, and so forth. This approach is error-prone, difficult to maintain, and in cases where the list contains hundreds or thousands of values, slows down query response time by many seconds. 
 
 A better solution is using the `search.in` function for security filters, as described in this article. If you use `search.in(Id, 'id1, id2, ...')` instead of an equality expression, you can expect subsecond response times.
 
 ## Prerequisites
 
-* The field containing group or user identity must be a string with the filterable attribute. It should be a collection. It shouldn't allow nulls.
+* A string field containing a group or user identity. It must have the filterable attribute. It should be a collection. It shouldn't allow nulls.
 
-* Other fields in the same document should provide the content that's accessible to that group or user. In the following JSON documents, the "security_id" fields contain identities used in a security filter, and the name, salary, and marital status will be included if the identity of the caller matches the "security_id" of the document.
+* Other fields in the same document should provide the content that's accessible to that group or user. In the following JSON documents, the "security_id" fields contain identities used in a security filter, and the name, salary, and marital status are included if the identity of the caller matches the "security_id" of the document.
 
     ```json
     {  
         "Employee-1": {  
-            "id": "100-1000-10-1-10000-1",
+            "employee_id": "100-1000-10-1-10000-1",
             "name": "Abram",   
             "salary": 75000,   
             "married": true,
             "security_id": "10011"
         },
         "Employee-2": {  
-            "id": "200-2000-20-2-20000-2",
+            "employee_id": "200-2000-20-2-20000-2",
             "name": "Adams",   
             "salary": 75000,   
             "married": true,
@@ -60,18 +62,19 @@ A better solution is using the `search.in` function for security filters, as des
     }  
     ```
 
-   >[!NOTE]
-   > The process of retrieving the principal identifiers and injecting those strings into source documents that can be indexed by Azure AI Search isn't covered in this article. Refer to the documentation of your identity service provider for help with obtaining identifiers.
-
 ## Create security field
 
-In the search index, within the field collection, you need one field that contains the group or user identity, similar to the fictitious "security_id" field in the previous example.
+In the search index, within the fields collection, you need one field that contains the group or user identity, similar to the fictitious "security_id" field in the previous example.
+
+1. Add a security field as a `Collection(Edm.String)`. Make sure it has a `filterable` attribute set to `true`. In this example, the security field is `group_ids`. It's populated with the string `["group_id1, group_id2"]`. 
+
+1. Set the field's `retrievable` attribute to `false` so that it isn't returned as part of the search request.
 
-1. Add a security field as a `Collection(Edm.String)`. Make sure it has a `filterable` attribute set to `true` so that search results are filtered based on the access the user has. For example, if you set the `group_ids` field to `["group_id1, group_id2"]` for the document with `file_name` "secured_file_b", only users that belong to group IDs "group_id1" or "group_id2" have read access to the file.
+1. Indexes require a document key. The "file_id" field satisfies that requirement. 
 
-   Set the field's `retrievable` attribute to `false` so that it isn't returned as part of the search request.
+1. Indexes should also contain searchable and retrievable content. The "file_name" and "file_description" fields represent that in this example.
 
-1. Indexes require a document key. The "file_id" field satisfies that requirement. Indexes should also contain searchable content. The "file_name" and "file_description" fields represent that in this example.  
+   The following index schema satisfies the field requirements. Documents that you index on Azure AI Search should have values for all of these fields, including the "group_ids". For the document with `file_name` "secured_file_b", only users that belong to group IDs "group_id1" or "group_id2" have read access to the file.
 
    ```https
    POST https://[search service].search.windows.net/indexes/securedfiles/docs/index?api-version=2023-11-01
@@ -87,23 +90,25 @@ In the search index, within the field collection, you need one field that contai
    ```
 
 ## Push data into your index using the REST API
-  
-Send an HTTP POST request to the docs collection of your index's URL endpoint (see [Documents - Index](/rest/api/searchservice/documents/)). The body of the HTTP request is a JSON rendering of the documents to be indexed:
 
-```http
-POST https://[search service].search.windows.net/indexes/securedfiles/docs/index?api-version=2023-11-01
-```
+Populate your search index with documents that provide values for each field in the fields collection, including values for the security field. Azure AI Search doesn't provide APIs or features for populating the security field specifically. However, several of the examples listed at the end of this article explain techniques for populating this field.
 
-In the request body, specify the content of your documents:
+In Azure AI Search, the approaches for loading data are:
 
-```JSON
+* A single push or pull (indexer) operation that imports documents populated with all fields
+* Multiple push or pull operations. As long as secondary import operations target the right document identifier, you can load fields individually through multiple imports.
+
+The following example shows a single HTTP POST request to the docs collection of your index's URL endpoint (see [Documents - Index](/rest/api/searchservice/documents/)). The body of the HTTP request is a JSON rendering of the documents to be indexed:
+
+```http
+POST https://[search service].search.windows.net/indexes/securedfiles/docs/index?api-version=2023-11-01
 {
     "value": [
         {
             "@search.action": "upload",
             "file_id": "1",
             "file_name": "secured_file_a",
-            "file_description": "File access is restricted to the Human Resources.",
+            "file_description": "File access is restricted to Human Resources.",
             "group_ids": ["group_id1"]
         },
         {
@@ -147,17 +152,11 @@ For full details on searching documents using Azure AI Search, you can read [Sea
 
 This sample shows how to set up query using a POST request.
 
-Issue the HTTP POST request:
+Issue the HTTP POST request, specifying the filter in the request body:
 
 ```http
-POST https://[service name].search.windows.net/indexes/securedfiles/docs/search?api-version=2020-06-30
-Content-Type: application/json  
-api-key: [admin or query key]
-```
-
-Specify the filter in the request body:
+POST https://[service name].search.windows.net/indexes/securedfiles/docs/search?api-version=2023-11-01
 
-```JSON
 {
    "filter":"group_ids/any(g:search.in(g, 'group_id1, group_id2'))"  
 }
@@ -186,7 +185,9 @@ You should get the documents back where `group_ids` contains either "group_id1"
 
 This article describes a pattern for filtering results based on user identity and the `search.in()` function. You can use this function to pass in principal identifiers for the requesting user to match against principal identifiers associated with each target document. When a search request is handled, the `search.in` function filters out search results for which none of the user's principals have read access. The principal identifiers can represent things like security groups, roles, or even the user's own identity.
 
-For an alternative pattern based on Microsoft Entra ID, or to revisit other security features, see the following links.
+For more examples, demos, and videos:
 
-* [Security filters for trimming results using Microsoft Entra ID identities](search-security-trimming-for-azure-search-with-aad.md)
-* [Security in Azure AI Search](search-security-overview.md)
+* [Configure document security for a chat app in Python](/azure/developer/python/get-started-app-chat-document-security-trim)
+* [Set up a security filter using Microsoft Entra ID](search-security-trimming-for-azure-search-with-aad.md)
+* [Set up optional sign in and document level access control (modifications to the AzureOpenAIDemo app)](https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/docs/login_and_acl.md)
+* [Video: Secure your Intelligent Applications with Microsoft Entra](https://build.microsoft.com/en-US/sessions/b5636ca7-64c2-493c-9b30-4a35852acfbe?source=/speakers/cc9b56a0-4af0-4b60-a2f3-8312c5b35ca2)