Skip to content

Commit 62c257e

Browse files
HeidiSteenHeidiSteen
committed
Renamed files and added content to ACL examples
1 parent 0b2593a commit 62c257e

File tree

15 files changed

+188
-67
lines changed

15 files changed

+188
-67
lines changed

‎Quickstart-ACL/requirements.txt‎

Lines changed: 0 additions & 9 deletions
This file was deleted.

Quickstart-Agentic-Retrieval/quickstart.ipynb renamed to Quickstart-Agentic-Retrieval/quickstart-agentic-retrieval.ipynb

File renamed without changes.

‎Quickstart-Agentic-Retrieval/requirements.txt‎

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,4 @@ aiohttp
44
ipykernel
55
dotenv
66
requests
7-
8-
--extra-index-url https://pkgs.dev.azure.com/azure-sdk/public/_packaging/azure-sdk-for-python/pypi/simple/
9-
azure-search-documents==11.6.0a20250505003
7+
azure-search-documents==11.6.0b12

Quickstart-Document-Permissions/document-permissions.ipynb renamed to Quickstart-Document-Permissions-Pull-API/document-permissions-pull-api.ipynb

Lines changed: 84 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -5,15 +5,57 @@
55
"id": "aba4346f",
66
"metadata": {},
77
"source": [
8-
"## Document Permissions in Azure AI Search"
8+
"# Document level access in Azure AI Search using the indexer pull APIs\n",
9+
"\n",
10+
"In Azure AI Search, you can use an indexer to pull content into a search index for indexing. This notebook shows you how index blobs that have access control lists (ACLs) in Azure Storage Data Lake Storage (ADLS) Gen2, and then query the index to return only those results that the user is authorized to view.\n",
11+
"\n",
12+
"The security principal behind the query access token determines the \"user\". The ACLs on folders and files determine whether the user has authorization to the content, and that metadata is pulled into the index along with document content. Internally, the search engine filters out any documents that aren't associated with the security principal.\n",
13+
"\n",
14+
"This feature is currently in preview.\n",
15+
"\n",
16+
"For an alternative approaching using push APIs to index any data, see [Quickstart-Document-Permissions-Push-API](../Quickstart-Document-Permissions-Push-API/document-permissions-push-api.ipynb).\n",
17+
"\n",
18+
"\n",
19+
"## Prerequisites\n",
20+
"\n",
21+
"+ Azure AI Search, basic tier or higher, with a [system-assigned managed identity](https://learn.microsoft.com/azure/search/search-howto-managed-identities-data-sources) and [role-based access control](https://learn.microsoft.com/azure/search/search-security-enable-roles).\n",
22+
"\n",
23+
"+ Azure Storage, general purpose account, with a [hierarchical namespace](https://learn.microsoft.com/azure/storage/blobs/create-data-lake-storage-account).\n",
24+
"\n",
25+
"+ Folders and files, where each file has an [access control list specified](https://learn.microsoft.com/azure/storage/blobs/data-lake-storage-access-control). We recommend group IDs.\n",
26+
"\n",
27+
"We recommend creating a virtual environment to run this sample code. In Visual Studio Code, open the control palette (ctrl-shift-p) to create an environment. This notebook was tested on Python 3.10.\n",
28+
"\n",
29+
"## Permissions\n",
30+
"\n",
31+
"+ On Azure Storage, **Storage Blob Data Reader** permissions are required for both the search service identity and for your user account since you are testing locally. You also need **Storage Blob Data Contributor**. This sample includes code for creating and configuring a container and blobs used in this demonstration.\n",
32+
"\n",
33+
"+ On Azure AI Search, assign yourself **Search Service Contributor**, **Search Index Data Contributor**, and **Search Index Data Reader** permissions to create objects and run queries. For more information, see [Connect to Azure AI Search using roles](https://learn.microsoft.com/azure/search/search-security-rbac) and [Quickstart: Connect without keys for local testing](https://learn.microsoft.com/azure/search/search-get-started-rbac).\n",
34+
"\n",
35+
"## Limitations\n",
36+
"\n",
37+
"+ Parsing indexer options aren't currently supported."
938
]
1039
},
1140
{
1241
"cell_type": "markdown",
1342
"id": "f445040a",
1443
"metadata": {},
1544
"source": [
16-
"## 1. Load Connections"
45+
"## Set up connections\n",
46+
"\n",
47+
"Save the `sample.env` file as `.env` and then modify the environment variables to use your Azure endpoints. You need endpoints for:\n",
48+
"\n",
49+
"+ Azure AI Search\n",
50+
"+ Azure Storage\n",
51+
"\n",
52+
"For Azure AI Search, find the endpoint in the [Azure portal](https://portal.azure.com), in the **Essentials** section of the Overview page.\n",
53+
"\n",
54+
"For Azure Storage, follow the guidance in [Get storage account configuration information](https://learn.microsoft.com/azure/storage/common/storage-account-get-info).\n",
55+
"\n",
56+
"## Load Connections\n",
57+
"\n",
58+
"Load the environment variables to set up connections and object names."
1759
]
1860
},
1961
{
@@ -32,11 +74,11 @@
3274
"# The following variables from your .env file are used in this notebook\n",
3375
"endpoint = os.environ[\"AZURE_SEARCH_ENDPOINT\"]\n",
3476
"credential = DefaultAzureCredential()\n",
35-
"index_name = os.getenv(\"AZURE_SEARCH_INDEX\", \"document-permissions-sample\")\n",
36-
"indexer_name = os.getenv(\"AZURE_SEARCH_INDEXER\", \"document-permissions-sample-indexer\")\n",
37-
"datasource_name = os.getenv(\"AZURE_SEARCH_DATASOURCE\", \"document-permissions-sample-datasource\")\n",
38-
"adls_gen2_account_name = os.getenv(\"AZURE_STORAGE_ACCOUNT_NAME\", \"documentpermissionssample\")\n",
39-
"adls_gen2_container_name = os.getenv(\"AZURE_STORAGE_CONTAINER_NAME\", \"documentpermissionssample\")\n",
77+
"index_name = os.getenv(\"AZURE_SEARCH_INDEX\", \"document-permissions-indexer-idx\")\n",
78+
"indexer_name = os.getenv(\"AZURE_SEARCH_INDEXER\", \"document-permissions-indexer-idxr\")\n",
79+
"datasource_name = os.getenv(\"AZURE_SEARCH_DATASOURCE\", \"document-permissions-indexer-ds\")\n",
80+
"adls_gen2_account_name = os.getenv(\"AZURE_STORAGE_ACCOUNT_NAME\")\n",
81+
"adls_gen2_container_name = os.getenv(\"AZURE_STORAGE_CONTAINER_NAME\")\n",
4082
"adls_gen2_connection_string = os.environ[\"AZURE_STORAGE_CONNECTION_STRING\"]\n",
4183
"adls_gen2_resource_id = os.environ[\"AZURE_STORAGE_RESOURCE_ID\"]\n",
4284
"token_provider = get_bearer_token_provider(credential, \"https://search.azure.com/.default\")"
@@ -47,7 +89,11 @@
4789
"id": "2d46b940",
4890
"metadata": {},
4991
"source": [
50-
"## 2. Create Index"
92+
"## Create an index\n",
93+
"\n",
94+
"The search index must includes fields for your content and for permission metadata. Assign the new permission filter option to a string filter and make sure the field is filterable. \n",
95+
"\n",
96+
"For local testing, `retrievable` can be **true**, but be sure to change it back to **false** if you make the solution available to others."
5197
]
5298
},
5399
{
@@ -66,8 +112,8 @@
66112
" fields=[\n",
67113
" SearchField(name=\"id\", type=\"Edm.String\", key=True, filterable=True, sortable=True),\n",
68114
" SearchField(name=\"content\", type=\"Edm.String\", searchable=True, filterable=False, sortable=False),\n",
69-
" SearchField(name=\"oids\", type=\"Collection(Edm.String)\", filterable=True, permission_filter=PermissionFilter.USER_IDS),\n",
70-
" SearchField(name=\"groups\", type=\"Collection(Edm.String)\", filterable=True, permission_filter=PermissionFilter.GROUP_IDS),\n",
115+
" SearchField(name=\"oids\", type=\"Collection(Edm.String)\", filterable=True, retrievable=True, permission_filter=PermissionFilter.USER_IDS),\n",
116+
" SearchField(name=\"groups\", type=\"Collection(Edm.String)\", filterable=True, retrievable=True, permission_filter=PermissionFilter.GROUP_IDS),\n",
71117
" SearchField(name=\"metadata_storage_path\", type=\"Edm.String\", searchable=True),\n",
72118
" SearchField(name=\"metadata_storage_name\", type=\"Edm.String\", searchable=True)\n",
73119
" ],\n",
@@ -83,7 +129,9 @@
83129
"id": "2b8945a2",
84130
"metadata": {},
85131
"source": [
86-
"## 3. Create data source"
132+
"## Create a data source\n",
133+
"\n",
134+
"Set the `IndexerPermissionOption` so that the indexer knows to retrieve the permission metadata."
87135
]
88136
},
89137
{
@@ -113,7 +161,9 @@
113161
"id": "ff5b912d",
114162
"metadata": {},
115163
"source": [
116-
"## 4. Get group ids"
164+
"## Get group IDs\n",
165+
"\n",
166+
"This step calls the Graph APIs to get a few group IDs for your Microsoft Entra identity. Your group IDs will be added to the access control list of the objects created in the next step. Two group identifiers are retrieved. Each one is assigned to a different file."
117167
]
118168
},
119169
{
@@ -136,7 +186,9 @@
136186
"id": "20588dc3",
137187
"metadata": {},
138188
"source": [
139-
"## 5. Upload sample directory and file"
189+
"## Upload sample directory and file\n",
190+
"\n",
191+
"This step creates the container, folders, and uploads the files into Azure Storage. It assigns your group IDs to to the access control list for each folder."
140192
]
141193
},
142194
{
@@ -179,7 +231,9 @@
179231
"id": "ca6de2ad",
180232
"metadata": {},
181233
"source": [
182-
"## 6. Run indexer"
234+
"## Run the indexer\n",
235+
"\n",
236+
"Start the indexer to run all operations, from data retrieval to indexing. Any connection errors or permission problems become evident here."
183237
]
184238
},
185239
{
@@ -210,7 +264,9 @@
210264
"id": "987dd496",
211265
"metadata": {},
212266
"source": [
213-
"## 7. Search sample data using x-ms-query-source-authorization "
267+
"## Search sample data using x-ms-query-source-authorization\n",
268+
"\n",
269+
"This query uses an empty search string (`*`) to provide an unqualified search. It returns the file name and permission metadata associated with each file. Notice that each file is associated with a different group ID."
214270
]
215271
},
216272
{
@@ -233,7 +289,9 @@
233289
"id": "c712ab8c",
234290
"metadata": {},
235291
"source": [
236-
"## 8. Search sample data without x-ms-query-source-authorization "
292+
"## Search sample data without x-ms-query-source-authorization \n",
293+
"\n",
294+
"This step demonstrates the user experience when authorization fails. No results are returned in the response."
237295
]
238296
},
239297
{
@@ -250,6 +308,16 @@
250308
"for result in results:\n",
251309
" print(f\"Path: {result['metadata_storage_path']}, OID: {result['oids']}, Group: {result['groups']}\")"
252310
]
311+
},
312+
{
313+
"cell_type": "markdown",
314+
"id": "e1ac3c84",
315+
"metadata": {},
316+
"source": [
317+
"## Next steps\n",
318+
"\n",
319+
"To learn more, see [Document-level access control in Azure AI Search](https://learn.microsoft.com/azure/search/search-document-level-access-overview)."
320+
]
253321
}
254322
],
255323
"metadata": {

‎Quickstart-Document-Permissions-Pull-API/requirements.txt‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
azure-identity
2+
aiohttp
3+
ipykernel
4+
dotenv
5+
requests
6+
msgraph-sdk
7+
azure-storage-file-datalake
8+
azure-search-documents==11.6.0b12

‎Quickstart-Document-Permissions-Pull-API/sample.env‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
AZURE_SEARCH_ENDPOINT=https://your-search-service.search.windows.net
2+
AZURE_SEARCH_INDEX=document-permissions-indexer-idx
3+
AZURE_SEARCH_INDEXER=document-permissions-indexer-idxr
4+
AZURE_SEARCH_DATASOURCEdocument-permissions-indexer-ds
5+
AZURE_STORAGE_ACCOUNT_NAME=
6+
AZURE_STORAGE_CONTAINER_NAME=
7+
AZURE_STORAGE_CONNECTION_STRING=
8+
AZURE_STORAGE_RESOURCE_ID=

Quickstart-ACL/acl.ipynb renamed to Quickstart-Document-Permissions-Push-API/document-permissions-push-api.ipynb

Lines changed: 50 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -5,15 +5,33 @@
55
"id": "810ce279",
66
"metadata": {},
77
"source": [
8-
"## ACL Quickstart for Azure AI Search"
8+
"# Document-level access example using the push document APIs\n",
9+
"\n",
10+
"In Azure AI Search, you can upload any JSON document payload to a search index for indexing. This notebook shows you how index documents that contain [user access permissions at the document level](azure/search/search-document-level-access-overview), and then query the index to return only those results that the user is authorized to view.\n",
11+
"\n",
12+
"The security principal behind the query access token determines the \"user\". The permission metadata in the document determines whether the user has authorization to the content. Internally, the search engine filters out any documents that aren't associated with the security principal.\n",
13+
"\n",
14+
"This feature is currently in preview.\n",
15+
"\n",
16+
"For an alternative approaching using indexers and pull API, see [Quickstart-Document-Permissions-Pull-API](../Quickstart-Document-Permissions-Pull-API/document-permissions-pull-api.ipynb).\n"
917
]
1018
},
1119
{
1220
"cell_type": "markdown",
1321
"id": "b6585426",
1422
"metadata": {},
1523
"source": [
16-
"## 1. Load Connections"
24+
"## Set the environment variables\n",
25+
"\n",
26+
"1. Rename `sample.env` to `.env`.\n",
27+
"1. In the `.env` file, provide a full endpoint to your search service (https://your-search-service.search.windows.net).\n",
28+
"1. Rename the default index name if you \n",
29+
"\n",
30+
"## Load Connections\n",
31+
"\n",
32+
"We recommend creating a virtual environment to run this sample code. In Visual Studio Code, open the control palette (ctrl-shift-p) to create an environment. This notebook was tested on Python 3.10.\n",
33+
"\n",
34+
"Once your environment is created, load the environment variables."
1735
]
1836
},
1937
{
@@ -30,7 +48,7 @@
3048
"load_dotenv(override=True) # take environment variables from .env.\n",
3149
"\n",
3250
"# The following variables from your .env file are used in this notebook\n",
33-
"endpoint = os.environ[\"AZURE_SEARCH_ENDPOINT\"]\n",
51+
"endpoint = os.environ[\"AZURE_SEARCH_ENDPOINT\", \"\"]\n",
3452
"credential = DefaultAzureCredential()\n",
3553
"index_name = os.getenv(\"AZURE_SEARCH_INDEX\", \"acl-sample\")\n",
3654
"token_provider = get_bearer_token_provider(credential, \"https://search.azure.com/.default\")\n"
@@ -41,7 +59,11 @@
4159
"id": "9327cf01",
4260
"metadata": {},
4361
"source": [
44-
"## 2. Create Sample Index"
62+
"## Create Sample Index\n",
63+
"\n",
64+
"The search index must includes fields for your content and for permission metadata. Assign the new permission filter option to a string filter and make sure the field is filterable. \n",
65+
"\n",
66+
"For local testing, `retrievable` can be **true**, but be sure to change it back to **false** if you make the solution available to others."
4567
]
4668
},
4769
{
@@ -59,8 +81,8 @@
5981
" name=index_name,\n",
6082
" fields=[\n",
6183
" SearchField(name=\"id\", type=\"Edm.String\", key=True, filterable=True, sortable=True),\n",
62-
" SearchField(name=\"oid\", type=\"Collection(Edm.String)\", filterable=True, permission_filter=PermissionFilter.USER_IDS),\n",
63-
" SearchField(name=\"group\", type=\"Collection(Edm.String)\", filterable=True, permission_filter=PermissionFilter.GROUP_IDS),\n",
84+
" SearchField(name=\"oid\", type=\"Collection(Edm.String)\", retrievable=True, filterable=True, permission_filter=PermissionFilter.USER_IDS),\n",
85+
" SearchField(name=\"group\", type=\"Collection(Edm.String)\", retrievable=True, filterable=True, permission_filter=PermissionFilter.GROUP_IDS),\n",
6486
" SearchField(name=\"name\", type=\"Edm.String\", searchable=True)\n",
6587
" ],\n",
6688
" permission_filter_option=SearchIndexPermissionFilterOption.ENABLED\n",
@@ -75,7 +97,9 @@
7597
"id": "f5cf4169",
7698
"metadata": {},
7799
"source": [
78-
"## 3. Connect to Graph to find your oid and groups"
100+
"## Connect to Graph to find your oid and groups\n",
101+
"\n",
102+
"This step calls the Graph APIs to get a few group IDs for your Microsoft Entra identity. Your group IDs will be added to the access control list of the objects created in the next step."
79103
]
80104
},
81105
{
@@ -98,7 +122,9 @@
98122
"id": "a9ce6d0f",
99123
"metadata": {},
100124
"source": [
101-
"## 4. Upload Sample Data"
125+
"## Upload Sample Data\n",
126+
"\n",
127+
"This step creates the container, folders, and uploads documents into Azure Storage. It assigns your group IDs to to the access control list for each folder."
102128
]
103129
},
104130
{
@@ -127,7 +153,9 @@
127153
"id": "e5c93f76",
128154
"metadata": {},
129155
"source": [
130-
"## 5. Search sample data with x-ms-query-source-authorization"
156+
"## Search sample data with x-ms-query-source-authorization\n",
157+
"\n",
158+
"This query uses an empty search string (`*`) to provide an unqualified search. It returns the file name and permission metadata associated with each file. Notice that each file is associated with a different group ID."
131159
]
132160
},
133161
{
@@ -148,7 +176,9 @@
148176
"id": "d31b67d8",
149177
"metadata": {},
150178
"source": [
151-
"## 6. Search sample data without x-ms-query-source-authorization"
179+
"## Search sample data without x-ms-query-source-authorization \n",
180+
"\n",
181+
"This step demonstrates the user experience when authorization fails. No results are returned in the response."
152182
]
153183
},
154184
{
@@ -163,6 +193,16 @@
163193
"for result in results:\n",
164194
" print(f\"Name: {result['name']}, OID: {result['oid']}, Group: {result['group']}\")"
165195
]
196+
},
197+
{
198+
"cell_type": "markdown",
199+
"id": "5ad253ec",
200+
"metadata": {},
201+
"source": [
202+
"## Next steps\n",
203+
"\n",
204+
"To learn more, see [Document-level access control in Azure AI Search](https://learn.microsoft.com/azure/search/search-document-level-access-overview)."
205+
]
166206
}
167207
],
168208
"metadata": {

‎Quickstart-Document-Permissions-Push-API/requirements.txt‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
azure-identity
2+
aiohttp
3+
ipykernel
4+
dotenv
5+
requests
6+
msgraph-sdk
7+
azure-search-documents==11.6.0b12

Quickstart-ACL/sample.env renamed to Quickstart-Document-Permissions-Push-API/sample.env

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
11
AZURE_SEARCH_ENDPOINT=https://your-search-service.search.windows.net
2-
AZURE_SEARCH_INDEX_NAME=acl-sample
2+
AZURE_SEARCH_INDEX=acl-sample

‎Quickstart-Document-Permissions/requirements.txt‎

Lines changed: 0 additions & 10 deletions
This file was deleted.

0 commit comments

Comments
 (0)