Skip to content

Commit cf690a1

Browse files
Merge pull request #228337 from HeidiSteen/heidist-sql
[azure search] Facet restrictions noted in the sql relational doc
2 parents 88b3566 + c95f7af commit cf690a1

File tree

2 files changed

+32
-15
lines changed

2 files changed

+32
-15
lines changed

articles/search/index-sql-relational-data.md

Lines changed: 20 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -8,31 +8,31 @@ manager: nitinme
88
ms.author: heidist
99
ms.service: cognitive-search
1010
ms.topic: how-to
11-
ms.date: 02/08/2023
11+
ms.date: 02/22/2023
1212
---
1313
# How to model relational SQL data for import and indexing in Azure Cognitive Search
1414

1515
Azure Cognitive Search accepts a flat rowset as input to the [indexing pipeline](search-what-is-an-index.md). If your source data originates from joined tables in a SQL Server relational database, this article explains how to construct the result set, and how to model a parent-child relationship in an Azure Cognitive Search index.
1616

17-
As an illustration, we'll refer to a hypothetical hotels database, based on [demo data](https://github.com/Azure-Samples/azure-search-sample-data/tree/master/hotels). Assume the database consists of a Hotels$ table with 50 hotels, and a Rooms$ table with rooms of varying types, rates, and amenities, for a total of 750 rooms. There is a one-to-many relationship between the tables. In our approach, a view will provide the query that returns 50 rows, one row per hotel, with associated room detail embedded into each row.
17+
As an illustration, we refer to a hypothetical hotels database, based on [demo data](https://github.com/Azure-Samples/azure-search-sample-data/tree/master/hotels). Assume the database consists of a Hotels$ table with 50 hotels, and a Rooms$ table with rooms of varying types, rates, and amenities, for a total of 750 rooms. There's a one-to-many relationship between the tables. In our approach, a view provides the query that returns 50 rows, one row per hotel, with associated room detail embedded into each row.
1818

1919
![Tables and view in the Hotels database](media/index-sql-relational-data/hotels-database-tables-view.png "Tables and view in the Hotels database")
2020

2121
## The problem of denormalized data
2222

23-
One of the challenges in working with one-to-many relationships is that standard queries built on joined tables will return denormalized data, which doesn't work well in an Azure Cognitive Search scenario. Consider the following example that joins hotels and rooms.
23+
One of the challenges in working with one-to-many relationships is that standard queries built on joined tables return denormalized data, which doesn't work well in an Azure Cognitive Search scenario. Consider the following example that joins hotels and rooms.
2424

2525
```sql
2626
SELECT * FROM Hotels$
2727
INNER JOIN Rooms$
2828
ON Rooms$.HotelID = Hotels$.HotelID
2929
```
30+
3031
Results from this query return all of the Hotel fields, followed by all Room fields, with preliminary hotel information repeating for each room value.
3132

3233
![Denormalized data, redundant hotel data when room fields are added](media/index-sql-relational-data/denormalize-data-query.png "Denormalized data, redundant hotel data when room fields are added")
3334

34-
35-
While this query succeeds on the surface (providing all of the data in a flat row set), it fails in delivering the right document structure for the expected search experience. During indexing, Azure Cognitive Search will create one search document for each row ingested. If your search documents looked like the above results, you would have perceived duplicates - seven separate documents for the Twin Dome hotel alone. A query on "hotels in Florida" would return seven results for just the Twin Dome hotel, pushing other relevant hotels deep into the search results.
35+
While this query succeeds on the surface (providing all of the data in a flat row set), it fails in delivering the right document structure for the expected search experience. During indexing, Azure Cognitive Search creates one search document for each row ingested. If your search documents looked like the above results, you would have perceived duplicates - seven separate documents for the Twin Dome hotel alone. A query on "hotels in Florida" would return seven results for just the Twin Dome hotel, pushing other relevant hotels deep into the search results.
3636

3737
To get the expected experience of one document per hotel, you should provide a rowset at the right granularity, but with complete information. This article explains how.
3838

@@ -42,7 +42,7 @@ To deliver the expected search experience, your data set should consist of one r
4242

4343
The solution is to capture the room detail as nested JSON, and then insert the JSON structure into a field in a view, as shown in the second step.
4444

45-
1. Assume you have two joined tables, Hotels$ and Rooms$, that contain details for 50 hotels and 750 rooms, and are joined on the HotelID field. Individually, these tables contain 50 hotels and 750 related rooms.
45+
1. Assume you've two joined tables, Hotels$ and Rooms$, that contain details for 50 hotels and 750 rooms and are joined on the HotelID field. Individually, these tables contain 50 hotels and 750 related rooms.
4646

4747
```sql
4848
CREATE TABLE [dbo].[Hotels$](
@@ -107,7 +107,7 @@ This rowset is now ready for import into Azure Cognitive Search.
107107

108108
On the Azure Cognitive Search side, create an index schema that models the one-to-many relationship using nested JSON. The result set you created in the previous section generally corresponds to the index schema provided below (we cut some fields for brevity).
109109

110-
The following example is similar to the example in [How to model complex data types](search-howto-complex-data-types.md#create-complex-fields). The *Rooms* structure, which has been the focus of this article, is in the fields collection of an index named *hotels*. This example also shows a complex type for *Address*, which differs from *Rooms* in that it is composed of a fixed set of items, as opposed to the multiple, arbitrary number of items allowed in a collection.
110+
The following example is similar to the example in [How to model complex data types](search-howto-complex-data-types.md#create-complex-fields). The *Rooms* structure, which has been the focus of this article, is in the fields collection of an index named *hotels*. This example also shows a complex type for *Address*, which differs from *Rooms* in that it's composed of a fixed set of items, as opposed to the multiple, arbitrary number of items allowed in a collection.
111111

112112
```json
113113
{
@@ -117,8 +117,9 @@ The following example is similar to the example in [How to model complex data ty
117117
{ "name": "HotelName", "type": "Edm.String", "searchable": true, "filterable": false },
118118
{ "name": "Description", "type": "Edm.String", "searchable": true, "analyzer": "en.lucene" },
119119
{ "name": "Description_fr", "type": "Edm.String", "searchable": true, "analyzer": "fr.lucene" },
120-
{ "name": "Category", "type": "Edm.String", "searchable": true, "filterable": false },
120+
{ "name": "Category", "type": "Edm.String", "searchable": true, "filterable": true, "facetable": true },
121121
{ "name": "ParkingIncluded", "type": "Edm.Boolean", "filterable": true, "facetable": true },
122+
{ "name": "Tags", "type": "Collection(Edm.String)", "searchable": true, "filterable": true, "facetable": true },
122123
{ "name": "Address", "type": "Edm.ComplexType",
123124
"fields": [
124125
{ "name": "StreetAddress", "type": "Edm.String", "filterable": false, "sortable": false, "facetable": false, "searchable": true },
@@ -132,17 +133,25 @@ The following example is similar to the example in [How to model complex data ty
132133
{ "name": "Description_fr", "type": "Edm.String", "searchable": true, "analyzer": "fr.lucene" },
133134
{ "name": "Type", "type": "Edm.String", "searchable": true },
134135
{ "name": "BaseRate", "type": "Edm.Double", "filterable": true, "facetable": true },
135-
{ "name": "BedOptions", "type": "Edm.String", "searchable": true, "filterable": true, "facetable": true },
136+
{ "name": "BedOptions", "type": "Edm.String", "searchable": true, "filterable": true, "facetable": false },
136137
{ "name": "SleepsCount", "type": "Edm.Int32", "filterable": true, "facetable": true },
137-
{ "name": "SmokingAllowed", "type": "Edm.Boolean", "filterable": true, "facetable": true },
138+
{ "name": "SmokingAllowed", "type": "Edm.Boolean", "filterable": true, "facetable": false},
138139
{ "name": "Tags", "type": "Edm.Collection", "searchable": true }
139140
]
140141
}
141142
]
142143
}
143144
```
144145

145-
Given the previous result set and the above index schema, you have all the required components for a successful indexing operation. The flattened data set meets indexing requirements yet preserves detail information. In the Azure Cognitive Search index, search results will fall easily into hotel-based entities, while preserving the context of individual rooms and their attributes.
146+
Given the previous result set and the above index schema, you've all the required components for a successful indexing operation. The flattened data set meets indexing requirements yet preserves detail information. In the Azure Cognitive Search index, search results will fall easily into hotel-based entities, while preserving the context of individual rooms and their attributes.
147+
148+
## Facet behavior on complex type subfields
149+
150+
Fields that have a parent, such as the fields under Address and Rooms, are called *subfields*. Although you can assign a "facetable" attribute to a subfield, the count of the facet will always be for the main document.
151+
152+
For complex types like Address, where there's just one "Address/City" or "Address/stateProvince" in the document, the facet behavior works as expected. However, in the case of Rooms, where there are multiple subdocuments for each main document, the facet counts can be misleading.
153+
154+
As noted in [Model complex types](search-howto-complex-data-types.md): "the document counts returned in the facet results are calculated for the parent document (a hotel), not the subdocuments in a complex collection (rooms). For example, suppose a hotel has 20 rooms of type "suite". Given this facet parameter facet=Rooms/Type, the facet count is one for the hotel, not 20 for the rooms."
146155

147156
## Next steps
148157

articles/search/search-indexer-howto-access-private.md

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@ When evaluating shared private links for your scenario, remember these constrain
5454

5555
+ An Azure PaaS resource from the following list of supported resource types, configured to run in a virtual network, with a private endpoint created through Azure Private Link.
5656

57-
+ You should have a minimum of Contributor permissions on both Cognitive Search and the Azure PaaS resource for which you're creating the shared private link.
57+
+ You should have a minimum of Contributor permissions on both Azure Cognitive Search and the Azure PaaS resource for which you're creating the shared private link.
5858

5959
<a name="group-ids"></a>
6060

@@ -142,7 +142,9 @@ When you complete these steps, you have a shared private link that's provisioned
142142
> Preview API versions, either `2020-08-01-preview` or `2021-04-01-preview`, are required for group IDs that are in preview. The following resource types are in preview: `managedInstance`, `mySqlServer`, `sites`.
143143
> For `managedInstance`, see [create a shared private link for SQL Managed Instance](#create-a-shared-private-link-for-a-sql-managed-instance) for help formulating a fully qualified domain name.
144144
145-
Other tools like the portal, Azure PowerShell, or the Azure CLI have built-in mechanisms for account sign-in. If you're using a REST client, such as Postman, you'll need to provide a bearer token that allows your request to go through. Because it's easy and quick, this section uses Azure CLI steps for getting a bearer token. For other approaches, see [Manage with REST](search-manage-rest.md).
145+
While tools like Azure portal, Azure PowerShell, or the Azure CLI have built-in mechanisms for account sign-in, a REST client like Postman needs to provide a bearer token that allows your request to go through.
146+
147+
Because it's easy and quick, this section uses Azure CLI steps for getting a bearer token. For more durable approaches, see [Manage with REST](search-manage-rest.md).
146148

147149
1. Open a command line and run `az login` for Azure sign-in.
148150

@@ -152,6 +154,12 @@ Other tools like the portal, Azure PowerShell, or the Azure CLI have built-in me
152154
az account show
153155
```
154156

157+
Change the subscription if it's not the right one:
158+
159+
```azurecli
160+
az account set --subscription {{Azure PaaS subscription ID}}
161+
```
162+
155163
1. Create a bearer token, and then copy the entire token (everything between the quotation marks).
156164

157165
```azurecli
@@ -168,7 +176,7 @@ Other tools like the portal, Azure PowerShell, or the Azure CLI have built-in me
168176
169177
1. Set the content type to JSON.
170178
171-
1. Send the request. You should get a list of all shared private link resources that exist for your search service.
179+
1. Send the request. You should get a list of all shared private link resources that exist for your search service. Make sure there's no existing shared private link for the resource and sub-resource combination.
172180
173181
1. Formulate a PUT request to [Create or Update Shared Private Link](/rest/api/searchmanagement/2020-08-01/shared-private-link-resources/create-or-update) for the Azure PaaS resource. Provide a URI and request body similar to the following example:
174182
@@ -197,7 +205,7 @@ Other tools like the portal, Azure PowerShell, or the Azure CLI have built-in me
197205
az account get-access-token
198206
```
199207

200-
1. To check the status, rerun the first GET Shared Private Link request to monitor the provisioning state as it transitions from updating to succeeded.
208+
1. Send the request. To check the status, rerun the first GET Shared Private Link request to monitor the provisioning state as it transitions from updating to succeeded.
201209

202210
### [**PowerShell**](#tab/ps-create)
203211

0 commit comments

Comments
 (0)